24.34 Self Hosted LLMs
Why Self-Host an LLM?
source: https://betterprogramming.pub/you-dont-need-hosted-llms-do-you-1160b2520526
- In the long run cost will be less compared to a hosted LLM API services.
- Initially, the cost of development will be less. However, when you scale in production the cost will increase linearly with the production load.
- But, in Self hosted LLMs, the upfront cost will be high, but in the long run the cost will be less in the production.
- Improved performance
- Privacy and Security
Challenges
- Complexity
