Solving the Semantic Layer:

Ben Schein: Welcome everyone to our webinar around solving the semantic layer, how we can use Forrester strategies with Domo technology. I'm Ben Schein, Chief Analytics Officer and SVP of product at Domo, and I'm very honored to have Boris Evalsion join us today from Forrester.

Boris and I have spent years now talking about data and semantics and BI platforms. And he brings really a wealth of knowledge that I know we use as we're crafting our product evolution strategy, and that benefits the entire industry. So with that, we're going to start off and turn it over to Boris to share some of his research and what he's seeing, and I'll come back a bit later and talk a little bit about how we translate that into Domo, into solving AI problems with our customers.

Boris Evalsion: Excellent. Ben and everyone, good morning, good evening, good afternoon, depending on where you're dialing in to listen to our joint webinar. Ben, thank you so much for allowing me and Forrester to share our research. Just like you, always very, very happy and interested in sharing our common knowledge because after all, we all serve the same clients.

So obviously we're going to start with some numbers. I wouldn't be a Forrester analyst if I didn't start with some numbers. Generative AI, agentic AI is it. It's the main topic of the day, and not for any reasons other than it is producing tangible benefits.

As you can see, about half of our clients are indeed claiming top line, bottom line, and risk avoidance type benefits. And I am extremely and especially pleased and interested that it's not just efficiency gains that we get from AI, but it is top-line benefits, like an increase in revenue. Something very tangible, something that will definitely be disrupting industries. But with generative AI, amazing disruptive capabilities comes a very interesting dilemma.

More and more of our clients are calling us and asking us, "Well, since generative AI, agentic AI, the co-pilots of the world seem to be able to do everything, why can't we just build our data and analytics solution?" And by the way, clients are not asking that specifically just about data analytics, but about all enterprise solutions, including automation. But since I sit in our data and analytics research team, those questions do come to me. So it is a very legitimate question. All of a sudden, buy versus build is back front and center of enterprise decisions.

And if there is one big screaming at you loudly from my screen, a key takeaway from our joint webinar is please, please, please, don't even start thinking about building any data and analytics enterprise grade. I'm not talking about a one-person business or some kind of a personal spreadsheet-based application, but large enterprise comprehensive data and analytics applications. Please, please, please do not even start thinking about it unless you have a very clear picture of what is it that you need to have under the covers vis-à-vis semantic layer, ontology, and knowledge graph. What are all of those things?

So let's start decomposing. So the big question is a bit of humor. To be or not to be, just like the famous Hamlet in the Shakespeare play pondered on. So to us, it's to build or to buy a BI or a data and analytics platform solution. Seemingly, agentic AI and generative AI are telling us, a lot of vendors, especially hyperscalers, are telling us you can just start talking to your data. As long as you have your data, you can just start talking to it. Wouldn't it be nice to finally be able to democratize data?

We've been talking about data democratization for decades, and we were sort of slowly but surely getting there. But the next step absolutely in data democratization is just being able to open up data access to everyone with close to zero training and technology savviness. Because who won't be able to ask questions like, "Who are my top five customers? What was my customer profitability last month?" And my personal favorite, and I'll explain why that is my personal favorite, "What was the sentiment of my customers listening to audiobooks on cruises?"

So we'll decompose that in a second. Seemingly it seems simple, but let's just peel one layer of the onion. When you ask, who are my top five customers? Top by what? Top by sales? Top by profitability? Top by loyalty? Top by volume? Top by web traffic? So all of a sudden, somebody needs to clarify to the analytical application what is it that Boris meant when he asked that question.

Then the next dilemma comes in a calculation like profitability. Profitability is notoriously difficult to calculate. There are academic books written on the subject. Years ago, when I was at PwC, we ran the whole project of how to help large clients calculate customer profitability. It's a whole science, and I bet you anything that your chief financial officer and your VP of revenue, and your chief marketing officer have three different opinions on how to calculate customer profitability. Profitability based on whose definition?

And then as I said, my personal favorite is the last question about customers listening to audiobooks on the topic of cruises or listening to audiobooks while on a cruise. Completely two different questions should generate completely two different SQL queries under the covers. This is one of the reasons, and we'll decompose this in a second, why is this so difficult?

But this difficulty of translating a natural language question into a legitimate enterprise grade where two plus two equals four, right? There should be no ambiguity. There should be no probabilistic outcome. This is one of the reasons that natural language to query or just talking to the data adoption is still relatively low because you need to solve for ambiguity. This is another great one.

Imagine you are a huge brand like let's say Walmart, right? And being Walmart, you actually sell apples, you sell groceries like apples, and you sell apples as iPhones. So when you say show me sales for Apple, is that Apple as a fruit or Apple as an iPhone?

Obviously, you need to have what's called schema awareness. If you ask for revenue, but in the database, the column is defined as sales, the system is not going to be able to answer that question unless you do what Ben and I are going to talk about in the next 20 or 30 minutes.

So we talked about disambiguation. We talked about context awareness. So where are all of these things happening? You are hopefully already beginning to get a sense that what seems to be really easy from a consumer point of view, because we all use generative AI today to answer an email or to maybe write a birthday poem to our friend or relative, is actually difficult. So please stop and think very hard about it.

So let's start now laying out what actually has to happen under the cover. Again, we already posed this hypothetical question. Wouldn't it be nice to just talk to data? It seems like I ask a question via a prompt. That prompt goes to some kind of a large language model. It generates an answer and presents it back to me. Easy? Well, not so fast.

Let's start adding all of the things that we just talked about. Where and how do you disambiguate? Where and how do you provide context? How do you ground the request in your enterprise data? And, very importantly, because LLMs have no concept of enterprise data security. If I sit in North America and I ask an LLM questions about Asia Pacific, and from my enterprise point of view, I have no access to Asia Pacific data, I shouldn't be looking at it. LLMs are not going to know about it. So you've got to build those guardrails.

Let's get into the thick of it. The very first thing that may be familiar to some of you, if you are a business intelligence pro, you've been dealing with semantic layers for a couple of decades. Let's start decomposing what a semantic layer does.

A semantic layer is what interprets the data. That is what interprets the reasoning. This is where the disambiguation is going to happen. This is what's going to be providing context awareness to the query. You cannot build a large, scalable, effective, efficient enterprise application without a rich semantic layer that has all of the above capabilities—reasoning, disambiguation, context awareness, et cetera.

This is where you define all of the aliases for customer. Customer may be a client. I may be asking about my buyers or purchasers or consumers, but at the end of the day, that's the customer. So no matter how you ask that question, the semantic layer should be able to interpret it.

Now, the semantic layer by itself needs to rest on something. It needs to rest on your enterprise definitions. Because in one company, a customer is a buyer. In another company, a customer is something else. In a medical profession, you don't even call it a customer, you call it a patient. So customer and a patient is the same thing. Somewhere you need to define not just aliases, but actually a formal definition of a customer or a buyer.

So now we get into your enterprise ontology, and we are already beginning to see some data structures, some taxonomies. Customer, here's a definition of a customer. What are typical attributes like name, phone, address, et cetera? It is a subclass of some kind of customer segment. So now we start building hierarchies.

This is very important because you may not be asking questions about a particular customer. You may be asking a question about a particular customer segment, certain demographics, certain psychographics, customer behavior, et cetera. And you start describing, again more in a qualitative way, how customer links to all of the other entities in your ontology, such as purchase orders or maybe an account manager in the B2B space to formalize those relationships. How does my customer relate to my products? Is it one-to-one, one-to-many? Who buys what?

Products don't buy customers, right? Customers buy products. So it's a directional link, probably in a many-to-many relationship. One customer can buy many products, and each product can be bought by many customers.

This needs to be instantiated by what we've been calling for years a knowledge graph. It is a structured graph-based representation of ontology. Semantics are based on ontology, and ontology is instantiated—ontology by itself is not going to be translated into a query. You need some kind of graph representation to finally build your query. When you ask that question, some logic has to go through all of these layers, and at the end of the day, build a query.

So here I represent it as a graph database. It doesn't have to be a graph database. It can be an SQL database because, in most cases, we still use relational databases. But basically, semantics resting on ontology instantiated via a knowledge graph is what's going to translate your natural language question into a query.

Here is the whole list of what you will need to build if you're building, and when you evaluate a platform like Domo for buying. These are the questions that you will need to ask Ben or his colleagues.

Now, this is for structured data. What we just went through is structured data, natural language to query. But obviously, before we start analyzing data, we need to find what we're looking for. You typically find something based on either a semantic or a vector search. A vector search increasingly is the more popular way to do this.

Bringing it all together is what we call a hybrid RAG—hybrid retrieval-augmented generation. This is how you contextualize and ground your enterprise data, metadata, security, semantics, ontology, et cetera. This is what you use to translate your natural language questions into legitimate queries and legitimate answers. All this needs to be built.

By the way, once you build this, this is the architecture that you start using for your gateways to both restrict what people can ask and validate once they get the answers.

Now, some of you may be getting a bit of smoke coming out of your ears thinking, "Boris, this is so complex, so difficult. This must be very expensive." I really need to clarify that this is nirvana. This is a future vision. Very few platforms today have all of the above, and you don't need to build this in a big bang approach. But in your technical architecture, in a two-year roadmap, you do need to plan for and account for all of these components. If you start building these components one by one and have an implementation plan that stretches out for a couple of years, that's perfectly fine. That's indeed what we are recommending. This is aspirational. This is a long-term vision.

The main point that I wanted to impress on everyone before you call us and say, "Boris, why can't we just build it?" is that the left side of this picture seems easy, but it's the right part of the picture that takes hundreds of thousands of person-hours to build.

Hopefully, this all made sense, and shows that this is not simple. Generative AI is going to disrupt how we talk to data and analyze data, but not before you invest in something like this. Ben, hopefully, this was useful to our mutual audience. I'll turn this over to you.

Ben Schein: Thank you so much, Boris. I don't know if I should be excited or exasperated and tired at the challenge that's ahead of us. But I think we know that there's a large challenge ahead of us to really make AI, generative AI, and agentic AI be real.

I know sometimes some of our engineering leaders almost wish, like, "God, why did ChatGPT have to happen?" because it set an expectation, which is something we can get to, but it's not as simple as just asking the question, as you said. As always, it's good to see what's going on in the industry. It's good to have your perspective. I think there's many points where I'm like, "Oh, okay. Well, we're not crazy. This is hard." These are things we need to be thinking about, even if it's not going to be a two-second solution.

We could talk more after and just chat a little bit about other challenges we're seeing and how it all comes together. I did want to spend just a few minutes talking about how we've been framing some of these challenges, some that Boris talked to a little bit, how we deal with some of the ancillary pieces around the diagram as well, and share Domo's perspective on how we've gotten here and where we're trying to go.

In a lot of ways, and I've used this really around the globe in different languages—not usually my language, it's a translator's language—this has been a very simple way for me to think about the things that have to happen to have a good generative AI, agentic AI, and it's really four parts. One of which Boris went deep on.

First, I need to pick an LLM. We can talk a little bit about how we approach that. Second, I need this ability to give it instructions, which is sort of part of a semantic layer in some extent because you need to be able to give guidance the same way you would a new intern or a new employee. Third, I need to provide knowledge, and that's where a lot of this, the ontology and the semantics come in, so that it can't just be vapid or just implied knowledge. We know LLMs will make up knowledge all on their own if you let them. They're going to go and create data or create a metric or a definition because you haven't given it to them. Finally, when we get more into the agentic space, we do need to make sure we can assign tools to what someone is allowed to do, what are the tools that let them sort of happen.

There's the bigger view of those things, and we use a worksheet with customers sometimes. Sometimes it's like, get out of the technology. You just need to answer some of these questions of what are the things, the knowledge, and the information. Then you have to go and say, "Oh, wait, do we even have that data? Do we have it stored? Do we have it accessible? Do we have it secure?" All those pieces.

I'm going to spend just a little bit of time on the left side here, which is a little bit outside maybe where we think about semantics, talking about LLMs, instructions, and tools. Then I'll go deeper into some of the knowledge side, how we're approaching it, what we have coming, and what innovations we want to continue to bring to our customers as they struggle with the challenges that Boris outlined for us already.

In terms of selecting the LLM, it's not everything. We know we can't just say we have an LLM and then we don't need any of the semantics or those other things. But it is important to have the ability and flexibility of how you select LLMs. I think back, it's hard to believe we just passed the three-year anniversary of ChatGPT, and in the months, so like January '23, a lot of our engineers started experimenting as everyone was. We saw this phenomenal shift in generative AI and conversational AI, and a lot of them really wanted to just start building, and that's great.

A lot of our early tech was on OpenAI because that's where a lot of the initial information was. We actually started to slow people down. We had to say, "Yes, we're excited. Yes, we want to test into this," and really lean into the idea of having more of an abstracted layer for our AI, which meant that we built intentionally in a way that lets you not just tie into a specific model. If we had built different features throughout the product, explicitly built around OpenAI, then when someone came and said, "I have my model in Databricks, or in Cortex, or Oracle, or Hugging Face, or wherever I'm hosting," it would require rewiring it. So we very explicitly built an abstracted layer that let us do that.

The other thing is a couple of years ago we did start providing what we call DomoGPT, which is a combination of models that we host. There are times when you need a custom model, like Boris was talking about medical. If you have very specific knowledge that needs to be in that large language model, you want to be able to bring your specific model. But it always makes sense theoretically, but practically, especially when you're testing, a lot of people just wanted a secure LLM that was good enough and basic, especially for a basic task. That's where providing DomoGPT within our cloud-based infrastructure—where our customers have already gone through all the security hoops, and we're already storing and accessing data for them securely with all of our multi-tenant security—actually was one of the big inflection points we saw. It wasn't just being able to hook into a model. Sometimes it was just having a model that could be quickly used for experimentation, to let people engage in a different way. That was step one: select the model.

Step two: how do we give instructions? More and more, we'll have places within our product—and I'll talk about it, this is in our workflow agent catalyst task—where, again, I'm training someone new. So part of it is how do I know the definition of metrics and the relationships and all that in the semantics, but also for this specific task or eventually this specific conversational agent or assistant, what is it that I'm asking them to do?

This is from an example we did with a fitness club. It's like, here's your role and expertise, here's your primary objective, here's your input. The same way, just like an intern or one of my product managers likes to say, it's like having a stubborn teenager who talks back to you; you need to be explicit about what your expectations are so that you can do that. More and more, in addition to the semantics side, we're finding ways in which you can configure and provide this context within the platform so that you have that.

And then the last piece before we get back to knowledge and semantics is being able to assign tools. This is from our workflow tool as well within Domo. There might be tools to do things in Domo, or tools to do things outside of Domo—an Asana, an Airtable, crawl a website, go to Google Analytics, send a message to Teams—all of those kinds of tools where, given the LLM I'm using and the context I've given it, you can try to solve the problem using these tools.

Also, a lot of that abstracted layer that we developed and made the investment in two and a half, three years ago, with specific services that can accomplish different tasks, has a little bit of that guidance already built in. Even if I have semantics, even if I have a custom LLM, even if I have instructions I've given it, sometimes there are things you don't want to have to do. You don't want to have to know all the nuances of text generation or specific things within Domo. We have an ever-expanding list of services that help take that LLM, those instructions, and that knowledge, and accomplish specific AI tasks. I think both of those work together in bringing it to life. So that's the left side of that chart.

But really, if we think about what Boris was sharing, and as we start thinking about where we go from here, this knowledge half, as Boris said, is crucial. You can have an LLM, you can have great instructions, you can even give it tools, and it's going to be garbage if what you're putting in is not that great. It has to be both structured and unstructured data, and we have both of those infrastructures now in Domo, which is a big part of our evolution.

This is also from our workflow agent, but I could add specifically structured data. Domo for many years has been really good at both getting structured data, but also empowering the last-mile user to enrich and make that data right and relevant to the business and to where they're sitting, closer to where things are happening. That's a big part of it: which data do I want to look at? And then also thinking about how do I provide unstructured data, which we have file sets within a Domo Documents feature that let me do that.

I want to unpack a few things about how I think about structured and unstructured data in ways that I think we differentiate as well. One thing, and Boris talked about this, is probably somewhere in the middle of his diagram. When you're providing knowledge, security is really important. Again, Domo has always excelled at governance, at both row-level security and then even now this year we introduced what I think is probably the most flexible, comprehensive column-level security hashing, hiding columns based upon user attributes.

Anything in our AI service layer, any of this work we're doing already knows who you are and if you have access to Asia-Pacific, like Boris said, or if I only am allowed to see a hashed version of the customer name. That's not necessarily the most exciting part of AI in general, nor is it the most exciting part of semantics and ontology, but if you get through all other things, you can have a great knowledge graph and all this knowledge, and then at the end of the day you say, "Yes, but when you deploy it, it's not going to have any security, or you're going to have to figure out how to honor your existing security." I don't think that is something that's going to be scalable at the enterprise scale. Maybe for 10 people, but not for hundreds and thousands of users who have to control that.

A second piece that we've been working on—and again, we still have to work at Domo to connect a lot of these pieces into a holistic semantic layer because the security layer has been there for a long, long time—is we've been adding, as we started experimenting with more of the chat interface using generative AI, this ability for users to have an AI dictionary about your data, being able to say what is the context. This is not just the definition; we've always had sort of a definition for a field in Domo, but that might be slightly different than additional information about how it's used, some of those things that Boris was getting at, as well as synonyms. If someone asks for outlay cost or someone asks for expenses, it's going to be this, or expenditure—these are all things we expect people to use, and if we hear that something's not working, we could go and adjust this. Much of this is back not with some ontology engineer or a data engineer, but with people who know what people are going to ask, empowering them to do that part of it.

Third on the structured data side is data models, relationships. We have done that work to start a data model within Domo where you could set those relationships, and also optimize the queries and other things. A lot of this is starting to come together. Boris said that's the ideal, what we laid out and what he laid out, but I think we're starting to get those pieces together that say: what do I need to do with structured data? Having the Domo documents and file sets to bring in unstructured data to the table, having this ability to provide context in specific situations, and bringing that all together in Domo in one place. We see limited adoption in certain places of different AI, but it's not nearly where it could be, especially with natural language. But even just in general, you're going to have a slice of people who are really leaning in. How do you make it easier and more reliable to do that?

Some of the things that I'm excited about that we're doing a lot of work on: first, you have AI readiness, you have definitions, you have security. We continue to think about how no one wants to have to go duplicate what they already set up in Data Virtuality or in Alation, or some other catalog. A lot of our thinking, as we think about models and the different AI dictionary and definitions, is how do we integrate with third parties, both cloud data warehouses. We also are one of the large partners of the Open Semantic Interchange, OSI, which Snowflake sort of led, but it includes Snowflake competitors and competitors of ours. Sometimes if you're going to make these kinds of things sort of common, you need to be able to work together. We are a part of this new effort, working to define a common language so that we can share semantic information from different warehouses and different tools, and have more of that so we're not repeating ourselves.

Even within our own software, if we have our magic ETL transformation, we're doing the work now to ensure whatever semantic information you have on an input, if it's still in the output, it carries that forward because you don't want to lose that. Or with our cloud connectors, which have been one of the core features of Domo for many years—connecting to Salesforce, connect to HubSpot, or connect to Adobe or whatever it is—if there's additional semantic information in those sources, making sure that when you connect, we're not just loading in the data, we're bringing in the semantics to help there as well.

Another piece in all this is we're looking more and more to combine schema, AI dictionary, and other relevant information into one place for management and viewing right within the dataset. Or even as I'm building data models that might include eight or 10 or 12 datasets, one place where I could see all the semantic information across that model and make adjustments. If there's a new synonym I need to add, I could see where does that go, where does it fit into the model, and all of those things. That will make it—again, we can have these features, but we want to make them easily accessible, easy to use, with multiple touchpoints, so that you can effectively manage that, because it has to be a living thing. You can't just set the definitions once and then you're good for five years. That's not how business works. That's not how knowledge really works in our world.

The fourth thing that I'm really excited about is this idea of custom assistants. Right now in Domo, we have an AI chat. We've had decent adoption, probably in that range that Boris was showing on the pie chart. Part of it is right now, when you're in an app in Domo or a dashboard in Domo, it sort of knows which datasets to look at. It has those AI dictionary items, which make it a lot better and more reliable, but it's still a little bit generic, and it doesn't have those other pieces of instructions and other knowledge and unstructured data we're going to provide to it.

The view of a custom assistant, similar to some of those screens I showed that right now are in workflows, is being able to say, "Hey, here are the instructions. Here's what kind of chat you want to do. Here are the explicit datasets I want you to use," not just based upon what you might be on a dashboard—it might use just the dataset from that dashboard, or it could use four that are relevant to having more of a chat with it. And then also unstructured data in addition to the structured data that comes in. I can sort of create this custom assistant that I could deploy in a dashboard or a Domo app, or deploy as its own thing even, and embed it somewhere else. We think that is going to be really exciting.

In a similar space, we're looking at how this all fits into the MCP protocol that's becoming more and more robust, so that similar to setting up a custom assistant to chat with, you could set up some of those same things in terms of what structured, unstructured data you want available via MCP endpoint, and what instructions you want there. Then, if you want to go use this integration or this tool within OpenAI because you have an enterprise OpenAI or enterprise Claude, or if you're building your own app and you want to be able to access it, you'll be able to leverage it in that way so it's not just stuck within the Domo ecosystem.

And the last one is, and Boris sort of called this out, is how do we make sure we have the right ontology and the knowledge graph technology wrapped around all of the semantics so that it actually comes to life at a scale that's effective, and that lets our customers unlock value from their AI journey. It's a lot to do ahead in 2026 and as we continue to build the product. But I definitely am excited about what we can do and some of the early signals we've seen when these combinations come together, when you have the right semantics, security definitions, AI dictionary, that you start seeing a different kind of potential. This is not just about saying, "Hey, checkbox, I have AI, and let me just go."

With that, I will stop sharing. We can maybe take a step back and just chat a little bit, Boris and I here, for a handful of minutes. We don't want to keep people too long-winded with us. I think we could probably talk for hours about this. I think first, Boris, you mentioned this is a multi-year process. How do you think people get started? It is overwhelming, right? It's a lot. Is there a place you would start, or how would you start goaling teams internally to use this if you know this isn't like a three-month project?

Boris Evalsion: So first of all, I always talk about two-week deliverables. It's like a Goldilocks and three bears challenge. You can't really deliver something in a couple of days. That's too short. But if you take more than a couple of weeks, if you start taking a couple of months, that's too long. So what's a perfect middle ground? We think over the years, we've observed this two-week period where using modern technologies, especially with agentic AIs, and if everything is resting on a solid semantic layer foundation, you can build a very robust prototype. There may be a bit of rubber band and scotch tape there, but you can build something very tangible and something that business users can touch and feel. So, first of all, plan for two-week deliverables.

Number two, prioritize what you can do in those two weeks. Don't prioritize high business value, high effort. Prioritize high business value for the lowest effort. Because you really need to build trust with the business users that this actually works and that this delivers business value. Measuring the business value of data and analytics is still an art. We can talk about how in the future it may become a science. Today, it's still an art.

There is no easy way to measure direct causational evidence that if you do this, your revenue is going to grow. We find lots of correlational evidence—measuring something before, after, et cetera. So it is very important to deliver something within a couple of weeks at a very low cost so that you build trust with your business users that this actually is providing value, and then you can go in and bravely ask for more budget for the next project.

But basically, I always talk about how no matter how many operational metrics and processes you track and run, at the end of the day, everything rolls up to no more than 10 or 15 C-level strategic metrics. No matter what industry and what business domain you're in, your CEO, when he or she walks into the office every morning, runs their business by no more than 10 or 15 metrics. Same for CFO, same for CMO. So you pick no more than one or two of those and say, "What can we do to democratize analysis, insights, consumption of those metrics via agentic AI?" Put together that project, deliver it, and then go to the next metric.

Ben Schein: Well, and I feel like having taken on, just a few months ago, the chief analytics officer role, now I'm sort of on both sides, and I'm trying to figure out how do we do that internally. Obviously, I always have been a big Domo user for over a decade now, both at Target and now at Domo. But I have to think about what are those two-week projects for us internally, as well as I really like that as a framework even for challenging our product teams of, as we're developing, of saying, "Did you build something that allows for value in two weeks if possible?" And if we've made something that the average user or a new user would take four months, especially in the AI space, it's like, maybe this might not be the right balance of usability and other things. And so I like that framework for pushing our teams.

One of the things I've observed too is sort of—and I mentioned a little bit around the chat and the dashboards—you have cases where at the same time, people are talking out of both sides of their mouth. They're like, "I want very specific. I want AI that's just focused on this one dataset that's in this one dashboard." And then they start using it, and they get frustrated because it doesn't have visibility to the other five things they care about. And so curious what you're seeing about best practices about how to sort of balance those two sides. Because if you just give everyone access to everything all the time, certainly without semantics, and even with a good ontology and semantic layering, it could get confusing. But how do you balance that need for focus, but also wanting the benefit or the potential of AI is I don't have to know what I want; it should look at everything that I have there.

Boris Evalsion: So you're beginning to touch on one of my favorite subjects. We started calling this ambient BI or ambient analytics, where insights should be coming at you, whether you're asking for them or not. And this is one of the use cases—not the only and not the most critical one, but definitely one of the use cases. So you're absolutely correct. Typically, I want to just get an answer to my question. I know what I'm asking for, but also, I don't know what I don't know.

So if when Domo provides me an answer to my question, somewhere on the side, very unobtrusively, but still kind of there, hovering over something in an ambient way, are some nudges. "Boris, you asked about A, B, C, but you know what? X, Y, Z may also be relevant." And it may be relevant because under the covers, Domo ran a machine learning algorithm that saw a very interesting key influencer on the variable that you just asked about. But you didn't ask about the key influencer. I, Domo, know about the key, and I'm going to gently tell you, "Would you like to know?" And it's not just machine learning, it's what my colleagues are doing.

So Boris, you've got colleagues in the same role, in the same position, in the same department. When they ask questions of this dataset, they also ask X, Y, Z question. Would you like to know? So gently nudging me without being too disruptive. That user interface, user experience is very important because I talked to one vendor that attempted to create this, and they said that users started immediately complaining that they started being bombarded with too much information—information overload.

UI and UX really need to be designed there very carefully, and each user should be able to pick, "I want to get X number of these nudges every day, but I want to get maybe a maximum of 10, but I don't want to get 100." Because you can go crazy with this. So I think that's—

Ben Schein: What's your nudge tolerance?

Boris Evalsion: Exactly.

Ben Schein: Also could be applied to marriage maybe at some point, but we can leave that—

Boris Evalsion: Right...

Ben Schein: ...to the side as well.

Boris Evalsion: Exactly. It's interesting because in some ways, we know what we think about semantics, and then about what a semantic layer looks like. But there is a whole part of it, especially at an enterprise scale, where your org structure is part of the semantics. Which customers you sell, what type of customers. So for Domo, it's like which region are you serving? What type of customers are you selling to? If you're selling to IT versus marketing, then other people with the same role selling to those personas, it's going to be much more relevant. That nudge is more relevant if you have that. And so it's really everything about your organization that can help you be more prescriptive in the persona you're serving that becomes part of the semantics.

Sort of on that note, one of the things I think, and I've gone back and forth. I spent many years with different tools at Target and other places where I felt like the architecture and the modeling was sort of a very technical feature. I had to go ask for it. And so sometimes I had that mindset of, "Okay, yes, it's a technical role to set this up." But the more and more I've learned and explored, it's like maybe to start, it's technical, but the real power comes when you can get business users to actually be part of that conversation, right?

So it's not like, oh, I had the conversation, well, we only need engineers to access the model. Well, no, they might start the model, but the power comes when, say, someone who's actually working in the warehouse can go in and make an adjustment to the semantics or at least submit a suggestion that gets approved. So what do you see in terms of involving more than just the technical folks in the development of that enrichment and semantics versus getting it out to people who maybe know the actual business problems better? How does that look in organizations?

Boris Evalsion: I'll elevate what you said even to a higher level, and I'll make a strong statement that may be somewhat controversial, but we almost see a direct correlation between more success in BI and data analytics when it's literally owned by a business C-level executive, and less success when it's owned by CIOs, CTOs. We do see traditional IT—IT as in infrastructure and software development—as a very different discipline than data and analytics. Obviously, data analytics rests on technical architecture and software development and infrastructure, but it's very different. IT may own software, but IT doesn't own data, right?

There's definitely anecdotal evidence that I can swear by because I talk to these people every day. And we also have in our surveys, every year in our annual survey, we let not even our clients, but a broad market, like a few thousand data analytics decision-makers out there self-assess themselves on their maturity levels, and they self-assess as beginners, intermediates, and advanced. And we do see a lot more higher maturity levels in those organizations where chief data and chief analytics officers report either directly to CEOs or maybe to COOs, but outside of IT. So, business ownership is absolutely critical.

What I always say, and especially today, with generative AI, and again, this is once again controversial. It's going to be a controversial statement, so treat it as very relative. Technology is easy, business is hard. It is much easier to take a smart, educated business professional and train him or her in technology because, Ben, you and I know, and we don't always talk about it, but under the covers, all of the analytical systems are Excel pivot tables on steroids. So if you're a business professional who understands how to use pivot tables, I can train you in Domo in less than a day, right? But you take the most highly educated technical professional and try to explain to them the intricacies of business...

I once dealt with a project with my former employer where we were trying to convince data modelers why in one of the ERP systems, they had 40 fields for invoice date. So invoice date on that particular ERP had 40 different fields. The date was printed, the day it was issued, blah, blah, blah, blah. The data modeler just couldn't fathom a business need for it, but there was a business need for that, right? So, more business ownership, more chances for success. Period, end of story.

Ben Schein: I think the answer is always whichever one shows the best result, you pick that date, right? So that's what everyone does. No, it is interesting.

I think if I look back at my time at Target, we did a lot of our innovation around data, including Domo and many other things. We started when we were a smaller team in the target.com and mobile business, when that was a little bit of an outcast. It was a very small percentage of the business, and we just were trying to figure out, like you said, the key metrics for that area, and driving at it from a BI side. There also were business things like recommendation engines and other things where there was real value from the data.

And then as it grew, we became more of a center of excellence and moved strategy and then COO, and then eventually did go back to IT, which I won't... Sometimes you go through these evolutions, but a lot of the real power is when you have it, or when you have it with the right support from IT, right? I don't think it can be entirely isolated, but it'll be interesting to see how we use AI to empower those teams in different ways and change some of those dynamics.

So I think last one maybe, we're getting close on time here, is just like, what else should we be thinking about as an industry? You mentioned ambient analytics. What do you think people are missing, Domo or others? Or just what do you think is going to come around the corner that we're not thinking about right now?

Boris Evalsion: Well, I'd like to double down and come back to my favorite topic of ambient analytics. I think we've exhausted the adoption of analytics via graphical user interfaces. We're tracking about 20% adoption, and I think that's where it's going to stay.

Again, I'm going to reiterate that the most user-friendly business intelligence platform is still Excel pivot tables on steroids. So there probably will not be more than 20% of business professionals who are comfortable with that kind of concept. So I think that that's where it will stay. I think we will increase adoption with natural language interfaces to data, probably add another 10 or 20% of professionals that will be able to talk to data. But then we also need to be building more ambient experiences that I split into two ways.

One, embedding insights anywhere I live. So if I live in an ERP system, that's where I need to get my insight delivered. If I live in Microsoft Teams or in Salesforce Slack, that's where I need my insight delivered. So I don't want to click out of that and go to Domo. I want a Domo agent sitting inside my Teams. If I spend my entire day, like we at Forrester do, basically talking to each other via Microsoft Teams, that's where I need to get my insights.

And the last one, very important, and I think that's going to be probably the majority of consumers of data, getting insights that they didn't even ask for, but rather by just subscribing to a particular dataset or a particular metric. So I'm a very busy high-level sales executive. I travel all the time. I wine and dine my customers. I don't have time to even talk to data. So even the natural language interface is too disruptive for me. But I want to track my customer behavior, customer profitability, trends in my customer behavior. So when a machine learning algorithm under the covers uncovers something interesting, I should be pinged. So I'm in the middle of a dinner, and I get a gentle nudge on my mobile phone. "Boris, something unusual is happening in your customer data. Would you like to take a look?"

Ben Schein: Yeah, and it's almost a nudge before the fire, right? Because right now you don't get nudged. You might not be looking at it because you're part of the 80% or the 60% that's not looking. And then, a week, two weeks, a month later, it's a huge fire to put out because the customer's mad.

And it's interesting, I've spent a lot of time internally with our teams of how do we look at performance across the Domo ecosystem? Often, we have a very robust cloud ops team that's looking at the total level and more at system stability. But often, it's not system stability. It could be my Adobe credentials expired, and the connector stopped working. That shouldn't require an engineer from our side. It also is probably not going to be enough, one customer, one set of their data, Adobe starting to error out. But if that's representing a lot of missed errors for 10% of this one customer success manager's book of business, they should know, right? They need that nudge because it does matter, and it's not about Domo. It's just, "Hey, did you know this was happening?" "Oh, great. Thank you so much."

Versus a month later, like, "Hey, Domo broke." "Well, your credentials ran out." "Well, no one told me." Those are the kinds of things where I think even from our own internal operations, we can see them, and it's a big goal, though, to get to that other 60%. Or, I think it's impossible to say I want to be as good as Excel. I don't think anyone is ever going to be as good as Excel, but it's a good challenge for all of us to think about the vast majority still, even after an LQ at your point, that we're not fully serving.

So anyway, with that, I'll end with a big thank you, Boris. This is always helpful for me to hear, I think, for our joint customers, like you said, to think about how they frame these questions, how they start solving some of these problems, and maybe in those two-week increments. And just really appreciate all your leadership in the industry and the value you bring to us and to all of our mutual customers.

Boris Evalsion: Thank you for having me. Always a pleasure.

‍