• Karel Bourgois, Founder & CEO Voxist,
• Paul Sweeney, Co-Founder and Chief Strategy Officer, Webio
• Lyle Pratt, Vida.io Founder & CEO
• Nikhil Gupta, Founder, CTO @ Vapi (YC W21)
• RJ Burnham, Founder & CEO at Consig AI

In 2020 we ran a panel at TADSummit on “Serverless and RTC (Real Time Communications).” The conclusions from that panel continue to ring true today, 4 years later.  The importance of examining the specific workloads, AGM (Amazon, Google, and Microsoft) define developer fashion, so serverless is required to remain relevant. The key is some workloads are well suited to serverless, and some like the real time communications core are not, even today. We brought together people working at the coalface of that segment of programmable communications. The result passed the test of time.

One of the current memes on LLMs (Large Language Models) is their impending disruption of enterprise SaaS. An AI agent is claimed to replace many enterprise SaaS, across customer communications, workflows, accounting, sales operations, enterprise resource planning, etc. Sebastian Siemiatkowski, CEO Klarna, announced earlier this year that Klarna’s new OpenAI-powered assistant handled two thirds of the Swedish fintech’s customer service chats in its first month. Not only were customer satisfaction metrics better, but by replacing 700 full-time contractors the bottom line impact is projected to be $40M. They’ve shut down SFDC and Workday SaaS. Klarna is an important data point, and as I’ve pointed out over the years BABS (Bay Area BS) can have a heavy spin.

When I examine what the innovators are doing, it’s highly focused, so the problem is constrained. Webio – debt conversations, Vida – specific SME customer conversations and scheduling, Voxist – I think is the broadest offer at the moment on capturing enterprise processes through voice. From academics I’m seeing discussions on the need to combine LLM + ML (Machine Learning) to deliver unique insights and fewer hallucinations. 

• The shift seems to be predicated on Agentic Architectures. What is an Agentic Architecture? What are the factors impacting its development?
• Most see Vertical LLM’s being the sweet spot, given the limited availability of enterprise data. Everything else gets eaten by the LLM Platforms (OpenAI, etc.). Are these assumptions correct?
• Where do you see the current Agentic Architecture successes? What are the numbers? What are the timelines, 2, 5, 10 years?
• How does LLM/Agentic change the specifics of what CX/ xCaaS does today? Voice, Messaging/ RCS/, eMail, Webchat = channels but what happens in and around these channels in this future picture.
• How does innovation in communications platforms translate into real value in AI and agent-based systems? Where do we see the most significant shifts occurring, and how do these technologies deliver beyond just the communications layer?

Summary of the Discussion

RJ opened the discussion with a position I think we can all agree with, SaaS will evolve to include AI/LLMs. It’s not only going to be LLMs, but other deep learning models, This is just the usual buzzword bingo, with a revolutionary spin, when it is simply evolutionary. However, as you’ll see there was much to discuss.

Karel made a great point, never believe claims X will be replaced by Y. For example, SMS will replace voice, or video will replace voice. Traditional PSTN voice has been declining for a more than a decade, but VoIP and embedded voice in devices like Alexa continues to grow. SMS has been declining, but IP messaging has grown more than SMS’s decline. The answer is always both and the pie gets bigger. RJ backs this up as choice enables us to use out time more effectively. Who would have thought a popular Alexa use case would be setting a timer while cooking in the kitchen.

X does not replace Y, rather together we’re able to do more.

An important framing that recurred across the discussion is its early days. The models are going to change, the reach of agentic architectures are going to change, whether an agent is helping a real person versus replacing a real person is going to change. We’re in a period of constant innovation, every month there’s something new. AI is a constant journey, not a product release, which is a challenge for the whole industry.

We get a little distracted on content consumption across age groups. Fortunately, Paul Sweeney brings us back on topic, something as simple as understanding a date, given all the formats that can be used is difficult. Using for example an intent engine was a significant unlock for gathering a date accurately within a conversation.

Moving to something complex such as meeting summaries. Often asked is, are the summaries any good? There’s a question to be asked before that question, who are the summaries for? What information is essential, can we be confident in the summary? There’s a brief discussion on our experiences of summaries, and ways to improve them.

Then Paul points out there’s also a temporal basis, has the context of the conversation changed? He gives the example of a debt conversation where the context changes to raise warning signs of an inability to pay. Paul is deftly steering the discussion onto where do summaries make sense? If the cause of that issue is healthcare related, then that becomes a higher priority then inability to pay. Hence there needs to be verifying control over how the agent manages the call. This leads to a discussion on a hierarchy of fuzzy rules. And we come back to control in LLMs at the end.

Lyle had been quiet so far, so a brought him into the conversation, and he made a great point that LLM’s do not kill SaaS, rather they bring the cost of intelligence close to zero for SaaS. Which means that Lyle’s developers do not need to code, people do not need to add to customer entries in CRM. There’s a massive wave of automation through LLMs used in SaaS.

I then moved the conversation onto vertical focus to limit data sets, does it still make sense. Lyle brought back RJ’s point, there’s no point specializing as the technology is developing too fast. Use what’s there to solve customers’ problems well enough. He then gave props to Mark Zuckerberg in open sourcing LLMs, which has changed the industry. We can avoid OpenAI ruling over industries or going bust.

Karel then brought an interesting point on the technology lock-in possible with speech to speech models. We discussed that with Rob Pickering, I will add that session soon. Karel gave an example with Microsoft and an Insurance firm that could not get an important use case working, in the French language. I think where language is an issue, e.g. French, we could see modularity as that will drive more sales. Especially when there is a political issue.

Lyle made a critical point, it is not the time to be a developer focused platform, that’s where the tech giants are focused. Rather focus on a market segment with a great experience, that is easy for the customer to buy and use. For the English language market that is 100% true. The platforms are getting simpler, and we’re only 2 years into this.

Paul again brought us back to the main topic of the conversation. There are several elements missing in LLMs, such as control/rational and orchestration. There was general consensus that the components we’re currently adding around LLMs will get built in. Karel gave a great example in healthcare that they have complex APIs, that an LLM could make very easy. Similarly in integrating an agent into a business, LLMs are going to make that easier.

We didn’t have time to discuss the work Anthropic and Microsoft are doing in desktop agents.

On what if OpenAI goes bust? It doesn’t matter as there’s open source?

But back to the opening question, Are LLMs about to disrupt enterprise SaaS, nope they’re about to make enterprise SaaS even easier to setup and use.

The insights:

• SaaS will evolve to include AI/LLMs
  • LLMs are about to make enterprise SaaS even easier to setup and use.
  • We’re entering a massive wave of automation through LLMs used in SaaS
  • X does not replace Y, rather together we’re able to do more.
• AI models are going to change. We’re in a period of constant innovation, every month there’s something new. AI is a constant journey, not a product release, which is a challenge for the whole industry.
• Props to Mark Zuckerberg in open sourcing LLMs, which has changed the industry. We can avoid OpenAI ruling over industries and not worry about them going bust.
• Control/rational and orchestration will get wrapped up into LLMs, it’s going to get even easier.
• There’s no point specializing as the technology is developing too fast, and we’re only a couple of years into it. Except where you can take advantage of language issues and local politics, e.g. in France.