So - I run a company that does AI agents for pharmacy. We are at Series B now and growing well - series C in 9-12 months given growth. We just acquired a competitor of that vendor so very much an insider.
First, The technology works, and it scales, but the whole bottleneck is domain expertise and implementation. These are expensive, and hard to scale. We hire pharmacists as project managers, that's how important domain expertise and implementations are.
Second, the amount of noise of "Voice AI for <industry>" is incredible. Most of them are completely clueless about the industry & basically "YC-striver" type who can only sell to other yc companies. They fail hard the moment they touch critical functions of the real world.
We have hundreds of pharmacies with us now and they mostly successful and happy. Our biggest friction is customers being burned by vendors like this and writing off AI altogether as "not ready".
> We have hundreds of pharmacies with us now and they mostly successful and happy. Our biggest friction is customers being burned by vendors like this and writing off AI altogether as "not ready".
This is such a great graph. Your company feels successful because your customer is happy yet the rest of the world would be just happy if your company died in a fire. This seems to also be just fine for companies like yours.
Pharmacy phone lines are a terrible first use case for voice AI. Older callers, drug names, insurance mess, zero patience. Automating "store hours" is fine. Automating refill exceptions is how you get this outcome.
I just don't get why we're not doing it vertically like you said here. Implement a part of the cognitive work using AI but let the hard parts be done by a person. They're still gonna be more rich, just less so.
Explain to me then why pharmacies have almost exclusively used IVR based phone systems for pharmacy refills and prescription management for the last decade?
Like they don’t even say the full name of the prescription they just tell you the first three letters and they’ll say “for the patient born on…”
Current pharmacy IT systems are currently almost 100% automated so how are you claiming it’s the first use?
Companies will pawn off thier customers to AI at their peril.
In many ways this is a repeat of the India call center train wrecks of the 00s. On paper, letting someone in Bangalore vs onshore handle incoming customer service calls looked like a path to amazing savings. In practice the customer experience was horrendous and companies CTRL-Zed these decisions and rapidly brought customer service back onshore again. AI is just that story of shortsighted decisions by weak leadership playing out all over again.
This, I suspect, is exactly like offshoring - and a lot of companies died during the offshoring fad because their software products become full of low quality merges and feature improvements that the company couldn't afford to fix or remove by the time they realized there was a problem. That remains my primary concern with AI code generation - well established codebases might become full of slop to the point of being unusable without a clear path back to maintainability leading to company death.
if you've tried to legitimately productionize LLMs to deliver a specific repeatable outcome, rather than human editable output(text, code), you know the models still aren't there yet and the whole product experience feels hacky
As someone who's been doing AI call centres for a while, I'm not surprised. It's really, really hard to build voice AI that works. There are no open source solutions that work out of the box.
But the biggest problem is ASR. WER is still atrocious even with SOTA models. When you add drug names and regional accents, it's a recipe for disaster.
The desired speed of AI adoption is what is hindering AI adoption. Big companies are trying to sell this as magic, and it's not. It requires proper use to get anything useful out of this system. But that takes away the magic of it and the investors can't have that as the eventual share price only works if it is magic. Frontier labs will smother their product with their timelines.
The idea, within regulated circles, is that it lacks provenance - there’s no vendor to sue. Many open source licenses also carry an explicit disclaimer of merchantability and fitness for purpose. These are taken seriously, and open-source software is viewed akin to car parts found on the side of the road. They could work but it’s better to let someone else verify.
But in the context of privacy it makes no sense though right? Open source and close source models (depending on vendor) both can be configured in a way that protects privacy of the user. The entire quote is wonky despite your point about companies preferring vendor-based system rather than raw dog oss
It will probably sound nice to laypersons but you're right, it has to have access to, at a minimum, a data extract with relevant information to perform it's tasks, although in my experience in the pharmacy IT world, a database view.
I think of this as the kiosk economy. It's like how businesses introduce kiosks even when having a human employee would be more convenient, just for the sake of greater profit.
The biggest problem with AI customer service is that a human employee would've let a minor issue slide without escalating it. But a chatbot often inflames the situation, and by the time the customer reaches a human agent, they're already furious.
Most people aren't rational or logical. Non verbal feedback, like acknowledging someone's anger and showing empathy, is incredibly important.
It's important to note while people are pumping the AI bubble that a near-moron with a checklist can do this job. They replaced AI with a regular old phone tree.
AI couldn't do this job with a checklist. It would forget the entire checklist after two responses, then get fixated on the words that the customer used to complain about how dumb the AI was and loop until they hung up and called back to hopefully clear the context.
edit: Unless Kinney had been sold a product that used caller ID to carry over context, then the customer's best chances would be to scream "Agent!" over and over again (to get put on hold for 45 minutes, then transferred to the most screamed-at Philippine minimum-wage gig worker on the planet), or to show up to the drugstore and start screaming at people.
So - I run a company that does AI agents for pharmacy. We are at Series B now and growing well - series C in 9-12 months given growth. We just acquired a competitor of that vendor so very much an insider.
First, The technology works, and it scales, but the whole bottleneck is domain expertise and implementation. These are expensive, and hard to scale. We hire pharmacists as project managers, that's how important domain expertise and implementations are.
Second, the amount of noise of "Voice AI for <industry>" is incredible. Most of them are completely clueless about the industry & basically "YC-striver" type who can only sell to other yc companies. They fail hard the moment they touch critical functions of the real world.
We have hundreds of pharmacies with us now and they mostly successful and happy. Our biggest friction is customers being burned by vendors like this and writing off AI altogether as "not ready".
> We have hundreds of pharmacies with us now and they mostly successful and happy
How happy are their patients?
Former patients
> We have hundreds of pharmacies with us now and they mostly successful and happy. Our biggest friction is customers being burned by vendors like this and writing off AI altogether as "not ready".
This is such a great graph. Your company feels successful because your customer is happy yet the rest of the world would be just happy if your company died in a fire. This seems to also be just fine for companies like yours.
Make it so the parts that are done by AI are rock solid, until then, they shouldn't be near a customer and be done by humans. Resist the greed!
Pharmacy phone lines are a terrible first use case for voice AI. Older callers, drug names, insurance mess, zero patience. Automating "store hours" is fine. Automating refill exceptions is how you get this outcome.
I just don't get why we're not doing it vertically like you said here. Implement a part of the cognitive work using AI but let the hard parts be done by a person. They're still gonna be more rich, just less so.
You figured it out, there, at the end!
Explain to me then why pharmacies have almost exclusively used IVR based phone systems for pharmacy refills and prescription management for the last decade?
Like they don’t even say the full name of the prescription they just tell you the first three letters and they’ll say “for the patient born on…”
Current pharmacy IT systems are currently almost 100% automated so how are you claiming it’s the first use?
I guess everything is ok these days as long as the President says "We screwed up [our most basic responsibility to customers], and we own that."
Companies will pawn off thier customers to AI at their peril.
In many ways this is a repeat of the India call center train wrecks of the 00s. On paper, letting someone in Bangalore vs onshore handle incoming customer service calls looked like a path to amazing savings. In practice the customer experience was horrendous and companies CTRL-Zed these decisions and rapidly brought customer service back onshore again. AI is just that story of shortsighted decisions by weak leadership playing out all over again.
Its still a huge and growing business and one that AI helps https://tech.yahoo.com/ai/articles/call-centres-using-ai-whi...
lol at the title
This, I suspect, is exactly like offshoring - and a lot of companies died during the offshoring fad because their software products become full of low quality merges and feature improvements that the company couldn't afford to fix or remove by the time they realized there was a problem. That remains my primary concern with AI code generation - well established codebases might become full of slop to the point of being unusable without a clear path back to maintainability leading to company death.
This is my favorite AI customer service story:
https://arstechnica.com/tech-policy/2024/02/air-canada-must-...
> Air Canada essentially argued that “the chatbot is a separate legal entity that is responsible for its own actions”
It's interesting that I had to close 3 popups just to read this article, two of them were for tracking cookies
Agreed. It's horrible and stupid. I actually run a service in the EU and I don't present that nonsense even with the risk of being sued.
GDPR popups remind me of the spam popups of the 90s.
No, maliciously compliant scumbag companies did that.
You don’t need a cookie banner for functional cookies, and opt out should be a single button.
GDPR requires websites to get user consent before doing sketchy shit. You don’t need a cookie popup if you’re not a scumbag.
if you've tried to legitimately productionize LLMs to deliver a specific repeatable outcome, rather than human editable output(text, code), you know the models still aren't there yet and the whole product experience feels hacky
Not to mention insecure.
As someone who's been doing AI call centres for a while, I'm not surprised. It's really, really hard to build voice AI that works. There are no open source solutions that work out of the box.
But the biggest problem is ASR. WER is still atrocious even with SOTA models. When you add drug names and regional accents, it's a recipe for disaster.
ASR = Automatic Speech Recognition
WER = Word Error Rate
Too many acronyms; not everybody is in your field.
Just curious, why are drug names such a disaster? Do LLMs not understand pronunciation keys?
The desired speed of AI adoption is what is hindering AI adoption. Big companies are trying to sell this as magic, and it's not. It requires proper use to get anything useful out of this system. But that takes away the magic of it and the investors can't have that as the eventual share price only works if it is magic. Frontier labs will smother their product with their timelines.
> Marraffa says Burt is fully HIPAA compliant and that the AI is not open-source and does not generate or manipulate data.
We need to fight the talking point that open source is bad!
How could something like open source ever be perceived as bad?
The idea, within regulated circles, is that it lacks provenance - there’s no vendor to sue. Many open source licenses also carry an explicit disclaimer of merchantability and fitness for purpose. These are taken seriously, and open-source software is viewed akin to car parts found on the side of the road. They could work but it’s better to let someone else verify.
But in the context of privacy it makes no sense though right? Open source and close source models (depending on vendor) both can be configured in a way that protects privacy of the user. The entire quote is wonky despite your point about companies preferring vendor-based system rather than raw dog oss
how can it not manipulate data and still work? does it do it by magic? absolute L quote by the spokesperson
It will probably sound nice to laypersons but you're right, it has to have access to, at a minimum, a data extract with relevant information to perform it's tasks, although in my experience in the pharmacy IT world, a database view.
but if the source is open, the ay-eyes can find the zero days! /s
Related: A pharmacy chain in Vermont implemented AI for efficiency (11 days ago, 40 points) https://news.ycombinator.com/item?id=49105190
Wow, it lasted over a week.
I think of this as the kiosk economy. It's like how businesses introduce kiosks even when having a human employee would be more convenient, just for the sake of greater profit.
The biggest problem with AI customer service is that a human employee would've let a minor issue slide without escalating it. But a chatbot often inflames the situation, and by the time the customer reaches a human agent, they're already furious.
Most people aren't rational or logical. Non verbal feedback, like acknowledging someone's anger and showing empathy, is incredibly important.
It's important to note while people are pumping the AI bubble that a near-moron with a checklist can do this job. They replaced AI with a regular old phone tree.
AI couldn't do this job with a checklist. It would forget the entire checklist after two responses, then get fixated on the words that the customer used to complain about how dumb the AI was and loop until they hung up and called back to hopefully clear the context.
edit: Unless Kinney had been sold a product that used caller ID to carry over context, then the customer's best chances would be to scream "Agent!" over and over again (to get put on hold for 45 minutes, then transferred to the most screamed-at Philippine minimum-wage gig worker on the planet), or to show up to the drugstore and start screaming at people.