Navigating AI Design Complexities

So, I'll try to do it really quick. My name is Kirill, I run Glassware Vagancy for the last nine years. We work with AI companies to help navigate through the user experience challenges. The good thing is that we occasionally go into AI design into 2017 when one of the clients hired us and said, hey, we have this amazing platform, can you help us to work on it? And right now we've been privileged to have a little bit more experience than others somehow.

Yeah, quick background on me. We have more than 100 different design awards. Our clients raised more than $1 billion on design work. I'm Ukrainian, moved to San Francisco. I've been nominated to the Agency of the Year three years back.

Okay, so how this whole thing started. We've been contracted to work on an interface for AI and SORCH. And if you think about AI and SORCH, it's obvious that you need to summarize a lot of data. And when you need to summarize a lot of data, you see that there's various amount of different types of data that you have to summarize. As you work on this project, you realize that the widgets might be an option. And the widgets might be the way how to navigate all these experiences because you have so many special data. And you can structure them in a really easy, intuitive way, just all together.

And then we kind of transformed this widget mentality to the bunch of different projects that we made. I just want to show you our emphasis on the way how we design products that you could just use tomorrow in your project right now. So our bet is we're in the middle of the transition. We would have this amazing future multi-model interfaces that will be there. But right now, everyone is doing chats. And so our chats are not always the cure. We believe that the cure might be as the prompt-based and graphic user interfaces. So it's essentially the mix of the interface where you analyze what patterns your users use. And then you transfer that into the widgets right inside of your navigation.

What good design is from our experiences is manual. You see what you're using inputs. You test it on a bunch of the users. You do it as specific as you can, depending on what type of the product you're building. And then you optimize it. And you're essentially into the loop where you, instead of you trying to predict the outcome, it's the AI who wants to work with the data and optimize the interface itself. So the future is essentially where AI is running multi-tests within your interfaces and creating the truly unique interface for every single customer. And there are some companies who are doing that. Right now, the ones called Cofray, they do this as the endless A-B tests on the text and images. But essentially, the idea is that you can fix and try to see what are the conversions would be if you change the whole interface right ahead.

Another thing I was, and I laughed when someone said it yesterday from that email app, I think it was Richard, that you can actually predict what users are going to be doing. And the more personalized data you have for each of the users, the easier for you to predict what they're actually going to be inputting into the product. Therefore, once you know that, you can build the widgets on top of the systems. Just think about it. If you would just implement the summarize button or act as button or find similar button in the chats right now, you could save hours and hours and months and years of the data.

So essentially, the whole concept is that we look at what prompts are people implementing in our products, and then based on that, we convert them into the shortcuts, into the small widgets that are built on top of the graphic interface. And it saves you endless amount of hours every single day.

How to do that? Well, you start with the comprehensive design system. Once you build the design system, you can essentially build any type of widgets out there. To then make them very model-wise so you can tweak it all the time. And here is the product that I was mentioning in the very beginning. Let's see. Yeah, we have this little demo.

So this product created for analysis, essentially for the venture capital and the hedge funds and the bank's analysis. If you want to invest into some of the markets, you're going to use this product because they collect a lot of the data that's available to the public. Or you can upload your own data sets into this analytics tool to essentially find the insights for the niche markets. Or maybe you talk to someone at VC and you say, well, this is a fascinating company, but I don't want to invest in them right now. So what this AI can do is that it essentially can follow the updates, see what they hire us for and send you a notification, hey, you might be interested in looking at them in the future.

But more commonly, this product is used for banks when you are looking at the different markets and you're working with a really large amount of data to create the reports. And then as business analysts, you actually work on very connected tasks. And we started to design this week, we were observing how people creating their analytics report. They were thinking, well, they're doing all the same, but for different verticals, for different companies, but it's very repetitive data they work. And we thought, well, why don't we build an interface that would help not only to get the data, but also create the reports itself.

And essentially, this is how we tweak the whole UX where instead of getting the data from our LLM, we essentially have a split screen that helps you to create those reports and send it to whoever you want. How? By doing simple and easy drop and drop interfaces to essentially collect it all together. But then we realized that the moment when analysis sends the report to some of the top management, you essentially get into the problem that this report is always outdated. And within the speed of the AI, you're always missing the latest updates on what's happening. So therefore, our model is built on updates, any numbers and evaluations, the currency updates every single time when you open the reports, you can keep and be updated.

Another cool stuff, if we think about the interfaces and the user experiences is that you have to create the products, or we tend to create the products that are very tailored to the experiences and very niche to the exact use cases that's happening in this industry. And if you think about it, analytics, you have this huge main point where the LLM gives you the output was just a text, and you need to transfer it into some kind of the charts. And therefore, go analyze and then within the one click, you can either do a table with or with the second prompt, you can actually transfer it into the bridges right ahead.

So we have to be sure that the AI can understand the context of the interaction and switch between the conversational and graphical interface. And I think that's the coolest thing that you can implement right ahead. But also you can give the user control over how you would like that personalization to be happening. And here it's hard to see, but the idea is that once you start typing something, you have a little bit of box and we are doing like our suggestion. And this is what's our habit to create that fractionalist experiences that we were talking about. So once you see that there is an outer suggestions happening while you're doing the research, you actually see the pattern. How can you use the product? And then you have the whole short case as the part of the our onboarding courses to see how you can leverage that right ahead into the product.

Another part that we figure out is that when you are working on some context, you might need to compare it with some of the documents or some of the resources that you made before. And therefore, with the click of a button, you can see the output and compare it to the exact document you have at the policy, all the strategies within your bank or venture fund. But also, over here, once you see some of the results, there are two things that people don't trust the AI. People are just like, well, where this information is coming from and why this is important. Therefore, we are building this cool call-outs that I really recommend you to build in your product. Why? Because we see that people actually start trusting our product much more once we create those calls right ahead. Because they know where the information is coming from. And even if it's come from their internal policies or even if it's come from the web, they can easily double click and see how they are connected.

Another part that helps us to grow the retention and the next part, even the domestic retention speech, is that we actually navigate the users and say, hey, here are all other things that you can do within this context that you're at. So we actually dive in deeper, showing you more functionalities that we already pre-designed for you because people don't know what you have created. And this is what Richard said at the beginning that, yeah, we are trying three versions of the product, but we never tried the pre-run if the pre-run is good enough. And this is how we actually show them, okay, you have some data and this is what else you can do with that.

So fighting against retention is one of the biggest challenges in the AI products in general because retention is not that great. Everyone is tracking that. In the AI product, people are open and they play into it. If they don't have an immediate gratification, they just close it and never come back and it's super hard to get back to them. Here's the lightly copyedited version of the transcript, preserving all original content:

So this is the very tough concept that we are working on right now, but I just want to share with you how we are trying to think about retention in an AI and a news channel. The idea here is that everyone is, you know, the typical user experience of you going through the news, you just read the news, you see the related articles, and then that's pretty much it. It's usually one-way type of communication. You read an article and while you have some thoughts, maybe you can share with someone, you can maybe send it or discuss it with someone, but you never really have a space to express yourself except for comments that are very rare in a big news website.

So what we thought is that why don't we create a portal that will help users to actually predict the outcomes of the news. So all the news that's happening in our life, you can interconnect them within one theme, and once they're connected within one theme, AI can actually predict what's going to be happening in the future. And users can vote on whether they agree on that or not agree on that, but on top of that, they can have the instant verification by giving some points or some kind of number that says how oracle they are regarding the different types of the news that's now there.

So what we're building is essentially the product that will help you to see the news, analyze and predict the outcomes to then get more points inside of this portal system and become the journalist if you wanted to create your own blog with this exact number that says how accurate you are in predicting the future. And this is how we plan to get much more traction compared to their typical regular news websites out there.

Final thoughts. I thought this would be really quick. The first one is the integration, and I think that's the future of the product design, because we essentially come into the data that are stored within the multiple products that are had to be interconnected in some way because this is how we can leverage the power of AI to create something truly unique. I can't wait for someone to create something like HTTPS to essentially all the AIs will be able to communicate with each other.

This is the stats about the retention and this is crazy. So this is the stats by SQL capital. They updated really lately. But the daily to active users are actually sinking down into AI products. So we're all into this vibe and a buzz that well, AI is going to change everything, but we have to solve the first problem with the daily active users and the monthly active users.

And how to do that? It's hard to present one of the last person, but asking questions and rephrasing the questions, we've seen the biggest spike within the retention of the products that we designed. So once you rephrase the question of the users, you ask much more in the details what they meant. You start to build this personal connection and people actually started trusting the product that you're building.

Yeah, onboarding process with that now we know educated friction where we show the part of the functionality, it stops the user from a typical flow, but you actually teach them how to use your product. And then, yeah, clearly communicate what AI is doing and why. It helps people to be more comfortable and to control the technology. Thank you.

Yeah, we take a couple questions till our break 11 and then we can go into quick break. But let's take some questions.

Do you see some difference on the retention on like different do you see the difference in the retention of the different platforms like mobile versus desktop versus like some others?

Well, interestingly, historically, most of our clients are actually desktop first products because they're heavily need to be sawed spaces. So I don't really have the data on that.

Robert, I want to add one. About the last product you showed the news product. For me, a problem to solve in years in general is like how you get more of a 360 degree view. You can kind of look at a story from different angles. Just interested in your decision making process to go down and sort of more of the gamification angle to make news more interesting and what led to that decision and which audience you were sort of targeting.

Yeah, so yeah, absolutely. Within the news concept, there are a lot of other cool AI features that we've added and especially that you can track the available and the public data through the different news sources. You can analyze to see if it's more centered right of the left who actually were created in this article. But also you can analyze the information on the dark net and comments in YouTube to see even more insights on what the people actually think about the topic.

And then you can combine this all together to see this, you know, how in crypto they have this grid level that says, well, you know, how really the market you are. You actually can have the same and that's what we're building the same bias level in general in and use. But then what you also can do is that you can not only understand whether it's led by the center, but you can also kind of send the context and the narrative that they're saying. Is it make you more angry? Is it, you know, is it built out of scarcity? And then you can hear this range of the emotions that will combine with the well, what party essentially pushing what agenda to give them non-biased information as much as you can.

Because I mean, from my perspective, every information is biased, but you can add that clarity and add some interesting features on top of it. And the communication part was just like, OK, so this is the basic level. But what can we do better to actually encourage users to share information and to stick with the product? And this is how we can build the whole notification system, the warning system on top of it. And this is how we can become a place where you actually marry Reddit-ish, news-ish AI product. And yeah, we'll see. We'll see how it goes. Thanks.

‍