How LlamaIndex is automating work of ~2 support engineers monthly

RESULTS

• ~2 support engineers saved by having kapa.ai implemented per month (assuming avg. 15mins per question, ~1000 questions per month, 160 working hrs per month)

• 125 days of end-user waiting time saved per month from instant answers (assuming avg. human response delay of 3hrs, ~1000 questions per month)

• 98% accuracy of questions answered by kapa.ai based on user-rated feedback

————

THE SITUATION

Rapid developer community growth puts a burden on support

LlamaIndex, an open-source project that provides a central interface to connect you LLM's with external data experienced explosive growth in its community, attracting more than 15,000 Github stars within a few months.

However, the surge in users also led to an overwhelming number of support questions, which puts a larger support burden on the core engineering team.

————

THE SOLUTION

An LLM-powered powered chatbot trained on LlamaIndex knowledge sources

Knowledge Sources

The kapa.ai worked with the Llama Index team to identify relevant knowledge sources to feed into a chatbot including +200 docs files.

Deployment Method

The chatbot was then deployed in the LlamaIndex Discord server in the #ask-kapa-gpt-index channel. Here it, over the course of four weeks, grew over 400% in the LlamaIndex Discord and went from answering around 90 questions a week to more than 400 questions a week as more and more community users became aware of the kapa.ai chatbot.

Try kapa.ai in the #ask-kapa-gpt-index channel on the LlamaIndex Discord.

————

THE RESULTS

Kapa.ai handles ~1k questions per month

kapa.ai's chatbot is a game-changer for LlamaIndex founder Jerry Liu and team. The chatbot has currently answered over 3400 questions with an estimated accuracy of 98%, significantly reducing the support hours needed to handle the LlamaIndex community. In fact, kapa.ai's solution has allowed LlamaIndex to reduce support hours by +1100 hrs, freeing up the team's time to focus on more strategic tasks, such as developing new features and improving the platform's performance.

The LlamaIndex community was thrilled with kapa.ai's solution. The chatbot provided quick and accurate answers to their most pressing questions, allowing them to use the tooling to its full potential. The reduction in support hours also means that the LlamaIndex community is receiving faster responses to their tickets, improving their overall user experience.