ALL POSTS

Why Adzviser won't follow the trend of self-hosting a large language model

An analogy of iPhone vs Nokia

When you purchase an iPhone, you're not just buying a phone; you're investing in an ecosystem. The App Store's vast array of applications transforms a standard device into a powerhouse of productivity and personalization. However, the iPhone may not fit perfectly in every hand — a compromise often overlooked for its superior capabilities. Will you consider buying a more expensive Nokia phone that fits your hands better but without the variety of apps offered in an App Store? The answer for most would be a resounding no - because almost no one wants to pay more for something less capable.

Our decision is no

The realm of retrieval-augmented generation has seen a surge in interest following the debut of ChatGPT. Innovators are eager to expand the capabilities of language models by incorporating real-time data such as weather updates, stock prices, or proprietary documents. Adzviser strives to provide accurate and real-time marketing data as a key plugin in the ChatGPT plugin store. This store, poised to evolve into the GPT marketplace, is to ChatGPT what the App Store is to the iPhone, significantly enhancing its fundamental capabilities. However, unlike some of its competitors, it will not

• host an open-source LLM that empowers a chatbot, emulating what ChatGPT already does
• or, call ChatGPT API on behalf of the users to build a wrapper around ChatGPT

Why Adzviser's decision is akin to choosing an iPhone over a Nokia

Capability Over Compromise. As of 11/09/2023, OpenAI’s latest GPT model has a 128K context or “more than 300 pages of text in a single prompt”. While open source LLM alternatives like Meta’s Llama2 or Claude 2 have their merits, they fall short when compared to GPT-4. Some may speculate that OpenAI's edge will diminish, echoing sentiments from a Google internal memo which stated, "We have no moat". However, this overlooks the significant financial resources — millions in GPU power and electricity — required to train such sophisticated models, a feat that currently only giants like Microsoft, who holds a huge stake in OpenAI, and Google can afford. Believing that open-source models will outpace GPT-4 is, at this juncture, optimistic at best.

Cost Efficiency Matters. The financial burden of hosting an open-source LLM in the cloud is not trivial. From personal experience, even with a powerful Nvidia RTX-4090 GPU and rented cloud GPUs from providers like runpod.io, the costs add up quickly. It's my firm belief that our users shouldn't have to pay more for an inferior chatbot service when a ChatGPT Plus subscription is both affordable and robust.

Addressing Potential Counterarguments.

• Security concerns - While some may hesitate to trust OpenAI with sensitive data, the same concerns apply to any LLM host. Without personal control over the model and data, security assurances are equally nebulous across providers.
• Customization - Marketers value data visualization, and while a custom LLM could theoretically offer more flexibility, my experience suggests otherwise. Moreover, with OpenAI's forthcoming custom GPTs—which will include a Code Interpreter—users will soon be able to visualize and analyze data more efficiently within a single ecosystem.

In essence, Adzviser won't be crafting an exorbitantly priced "Nokia" in an "iPhone" world. The choice is clear: why pay more for less?