Skip to main content

Build a LEGO Chatbot

· 2 min read
Daniel Fang
Daniel Fang
AI, Robotics & LEGO Enthusiast

In my latest experiment with AI-powered robotics, I decided to take all the API help specs from the Spike Prime 3 user guide and load them into Cosmos DB as a knowledge base. The goal was to use this data to generate better, more targeted Python code for Spike Prime robots using a LangChain RAG integration.

Here's the plan I followed:

  1. Extracting API Data: I pulled all the relevant API definitions and code snippets from the Spike Prime user guide. This included functions related to movement, turning, and sensor control.
  2. Cosmos DB as a Knowledge Base: I uploaded the extracted API documentation into Cosmos DB, setting it up as a searchable knowledge base. This would allow us to easily retrieve API definitions and related examples.
  3. LangChain Integration: The idea was to use LangChain's search capabilities to query Cosmos DB. The chatbot would then fetch relevant Python API definitions and example code snippets directly from the knowledge base.
  4. Generating Python Code: Once the relevant information was fetched, the chatbot would combine this with user input to generate Python code that could control the Spike Prime robot.

However, I ran into an unexpected issue: Despite having all the Spike Prime API information in Cosmos DB, the chatbot still ocassionally attempted to rely on external Python libraries from the internet instead of using the custom API definitions I had loaded. This led to code that wasn't compatible with the Spike Prime system, as those external libraries were not designed for LEGO robots.

This is a challenge I need to address. The chatbot is prioritizing internet-based solutions over the internal knowledge base, which defeats the purpose of having a custom-built, Spike Prime-specific AI assistant.

Next steps? I plan to tweak the LangChain logic to ensure the chatbot prioritizes fetching from Cosmos DB and only uses the API definitions we've provided. This will help ensure that the Python code generated is compatible with the Spike Prime robot. Stay tuned for updates on how I solve this!