After reading Andressen Horowitz’s article on LLM Architecture I decided to look into Langchain. Langchain describes itself as a framework for developing applications powered by language models. I have been working with OpenAI for some time now

❓Why I decided to look into Langchain

1. I’ve been working with OpenAI for some time now and I was intrigued that there are already frameworks for building applications powered by language models.
2. One of the problems I’ve been working on is effectively testing and iterating models and I hoped that Langchain would help with that.
3. We’re putting our models in production and we want to improve the performance of our models. I am anticpating increased complexity and I expected langchain to provide an opinionated way to manage that complexity.

🔗Langchain as a framework

Langchain seems to have 2/3 languages they support; javascript/typescript and python. I looked at the python version because I prefer to use python over javascript for data science stuff. Right now we use a mix of Go and Python but I was open to switching to python if it made sense.

🔑What I did

1. I setup a fastAPI project and made an endpoint for a simple question answering model.
2. I setup a langchain project and tried to use the components to make a simple question answering model.
3. I integrated it with our existing WeOS chat app to see how it would work in production.
4. I setup an Agent
5. I used some of the tools and toolkits

👍What I liked

1. Setup was easy
2. The documentation was good with clear examples. The different components were well explained and I was able to get a good understanding of the framework
3. There are a lot of people using langchain so you will find articles to help you along the way
4. The framework and examples give you ideas of how you can structure your own prompts ( this example shows you how to make a custom agent but the example prompt is also useful)
5. The design pattern of the framework is easy to use and you can really chain together different components to build your own implementation.
6. It uses a MIT license so we would be able to use it in production
7. I liked that it was LLM agnostic and you could mix and match

👎What I didn’t like

A lot of what I didn’t like is related to python not being my main programming language (I main Go) and the reasons for that

1. It was a bit confusing on whether to use a chain or an agent. I ended up making a chain and linking that to an agent
2. The method signatures are similar between components but different enough that I had to keep looking at the documentation to see what I was doing wrong.
3. It was really slow compared to our existing implementation. I’m not sure if this is because of the way I implemented it or if it’s because of the framework.
4. It does mask some things (this is common with frameworks) and I had to dig into the code to see what was going on. I’m not sure if this is a good or bad thing.

🏁Conclusion

Langchain is a treasure trove of best practices that is extremely helpful. We’re pretty far along with our own AI implementaitons so switching from Go to python would be an investment and the speed concern is a blocker. We’ll most likely be taking the links from Langchain and using them in our own implementation. I would recommend Langchain to anyone who is starting out and looking to go from 0-100 really quickly.