Creating an AI Chatbot for Technical Queries

In July 2023, I embarked on an exhilarating one-week journey to create an AI chatbot using Python. I worked with a team spread across multiple geographical Darwin locations. The challenge was daunting, but the team's enthusiasm was contagious. With limited time, we knew we had to be efficient and focused to deliver a functional and robust chatbot. In this blog, I'll share the experiences, insights, and triumphs of our week-long adventure.

The project began with an intense brainstorming session to define the chatbot's purpose and functionality. We identified key features, such as the large language model, response generation, and integration with external APIs. Our choice of Python as the development language proved to be a wise decision, given its extensive libraries and frameworks supporting AI development. With the roadmap laid out, we split the tasks among team members and dived straight into the code using an Agile Kanban approach.

I was assigned to the API development for the Chatbot. The goal was to be able to use a private chat completions API API that connected to the ChatGPT Large Language Model to provide a response for a technical query. A large Language Model works like the auto complete that you might find on mobile device keyboards. It takes an input string (in this case a question), predicts what text would follow after the input (in this case an answer), and returns an output string (the answer). My implementation would hook up to a REST API created by another developer to be able to post HTTP queries and retrieve responses.

Well, as simple as it sounded, the responses were… weird.

Thankfully, after thorough research and determination, I learnt about Large Language Model Context. LLMs need context to be able to tailor and format their answers to a specific use case. For the requirements of this project, we needed the Chatbot to have a conversation with the user, akin to ChatGPT.

Therefore, I needed to use an abstraction of the LLM called a ChatLLM. A Chat Large Language Model is a special variant of an LLM that expands its capability. It is able to take a list of messages and return an output list of messages. Therefore, it can hold a conversation. It is provided with the context and formatting logic to provide a coherent, natural language response.

As the week came to an end, we gathered to showcase our AI chatbot to the wider team. There were elements of excitement and nervousness, but the presentation went smoothly. The week of relentless work and thorough research had paid off. We received valuable feedback and suggestions, which fueled our enthusiasm to continue refining and expanding the chatbot.

Building an AI chatbot in a one-week sprint using Python was an exhilarating experience that pushed my limits as a software developer. I witnessed the potential of AI technology and the power of Python in transforming ideas into reality.

The journey taught me valuable lessons in teamwork, adaptability, and the rewards of facing challenges head-on. As we look back on this whirlwind week, we carry with us a newfound sense of accomplishment and an eagerness to embark on even more ambitious projects in the future. The importance of scoping and planning, was clear to me as the project progressed. There truly is immense value in collaboration and continuous learning.