API cost – How to provide free prompts? – API

I am building an app which would integrate ChatGPT and other models. We’re planning on offering a free version but it seems OpenAI charges for every token. How do apps provide a free service using OpenAI models when they are charged? Any ideas how they manage to?

Like how it often works …with wealthy investors that are prepared to take a hit and pay the bills up front to get market share, then pivot to a freemium service later on …

You can always implement a quota system so your downside risk is controlled.

My guess would be that they use deep pockets from investors like @merefield said, but also that they may take advantage of some of Microsoft’s startup grants.

$2500 isn’t anything to scoff at, especially if you use gpt-3.5-turbo-0125 with the new lower pricing.

Thanks @merefield and @trenton.dambrowitz .

I think there must be a trick. The bills are just too high.

Sadly no tricks, just market share capture by loss leader. It’s a form of calculated gambling/investing and it’s very common in emerging markets.

There might be tricks to decreasing your cost per prompt though, are you willing to share more details on the service you intend to offer and the prompts you’re using?

It’s a platform which integrates several models for general purpose users of Gen AI. We were planning on a fermium model but the economics of it don’t make sense yet. Not to spam users here with personal topics, plz let me know if you’re happy yo have a private chat and how.

You would be going up against the likes of Poe and Nat.Dev, both of which are funded by billionaire primary investors, if you can find a way to make it work that would be awesome, but I think you need to propose a value add rather than simply access. You can justify a 10x premium if your application does something that saves time or money or is simply entertaining, but it’s a very tough ask to request payment for a service that is not substantially different to what others provide, I wish you well.

Super insightful, thanks @Foxabilo.

Is there a way of integrating apps like ChatGPT and using their free service without incurring API costs? As if each user of our app is a user of ChatGPT?

That is against the terms of service of ChatGPT.

See “What you cannot do”

That is the reason ChatGPT has so much browser security and checking being done with it now, when it first launched that was being done by many people and it placed a huge load on the system. It is “actively” discouraged by security and legal teams.

I understand. But I didn’t mean any breach of T&Cs. What I meant is, since OpenAI allows free usage up to a limit, do they care where users come from? Assuming they get users’ credentials and other requirements, do they care if users come from 3rd party apps?

Firstly, it doesn’t pass the common sense check.

Secondly they state you cannot:

“Automatically or programmatically extract data or Output (defined below).”

If you can’t do that, as they prescribe, you cannot integrate it into your system.

Therefore you must use their API which has a more appropriate set of conditions attached that permits such use.

And seriously, if you are so desperate that you have to start hacking ChatGPT output, you haven’t got a serious business case and should rethink it.

My suggestion is:

• Use the API
• Limit the volume in some way so you can bear the cost of initial phase
• Have a plan to evolve the pricing
• Evolve the pricing levels as you scale so everything balances.

And/or find a sufficiently wealthy investor.

But smart investors will, quite rightly, run away from hacked solutions!

If you’re you’re super confident in your idea you could always just look for an investor.

There are a few “tricks” going on here:

1. The use of the API may be a marketing expense. Some companies, even small ones, have marketing budgets in the millions of dollars.
2. The use of the API may be a “test and get feedback” research investment. If the company is a funded effort to build a product, this may be worth it.
3. The cost to a human being who also needs to pay the rent and food and upbringing of their children, may seem high, whereas it’s not particularly high in the context of “running a business.”
4. The US is a high-cost country, so something that might seem affordable with in the context of the US, may seem expensive to someone in a lower-cost country.

“Funding” can come from many places. It doesn’t have to be a traditional venture capitalist. Some people take out a loan against their house. Some people use their savings account. Some people raise money from friends and family.

that’s business economics, nothing specific to chatgpt.
You do you calculations right and your free version of the product (how much it will cost) and what percentage from the free will move to paid.
Based on the numbers you will need to calculate what you need to become profitable.

Get angel investors like OpenAI did. everything has a cost. Even yourself.

Did you try asking Chat GPT this question?

I did.

Here’s an answer:

Offering a free version of your app while integrating OpenAI models like ChatGPT can be challenging due to the costs associated with token usage. However, there are several strategies and considerations you can explore to manage these costs:

1. Limit Usage: Implement usage limits for free users. Set a cap on the number of tokens they can consume within a certain time frame. Once they exceed this limit, you can prompt them to upgrade to a paid plan or wait until their usage resets.
2. Optimize Requests: Minimize unnecessary requests to the OpenAI API. Implement caching mechanisms to store responses for frequently asked questions or repetitive queries. This reduces the number of API calls and conserves tokens.
3. Prioritize Features: Offer a basic version of your app with limited functionality for free users. Reserve advanced features, such as longer conversation sessions or access to additional models, for paid subscribers.
4. Monetize Through Upgrades: Provide premium tiers with enhanced features or increased usage limits. Users who require more tokens or advanced functionality can choose to upgrade to a paid plan, helping to offset the costs associated with token usage.
5. Seek Sponsorship or Partnerships: Explore opportunities for sponsorship or partnerships with organizations that may be interested in subsidizing the cost of token usage in exchange for visibility or access to your user base.
6. Utilize OpenAI’s Pricing Programs: Keep an eye on any pricing programs or initiatives offered by OpenAI that may support startups or projects with limited budgets. They may offer discounts, grants, or other forms of assistance.
7. Diversify Revenue Streams: Consider alternative revenue streams, such as advertisements, affiliate marketing, or offering premium content or services within your app, to supplement the costs associated with token usage.
8. Monitor and Adjust: Continuously monitor usage patterns and costs associated with token usage. Adjust your pricing strategy, usage limits, or feature offerings accordingly to ensure sustainability.

By implementing a combination of these strategies, you can provide a free version of your app while effectively managing the costs associated with integrating OpenAI models like ChatGPT.