MotokoPilot: Harnessing AI to Empower the Motoko Development Community

Originally published at https://icpcs.medium.com on June 22, 2023.

In an era where AI is making profound strides across various domains, software development is no exception. The emergence of AI-powered tools has proven to significantly boost developers’ productivity, streamline the learning process of new languages and frameworks, and enhance the overall development workflow.

Despite these advancements, the applicability of most large language models (LLMs) when it comes to lesser-known languages, such as Motoko, is somewhat restricted. This is primarily due to the lack of representation of Motoko in datasets used to train these models. As a response to this limitation, the ICPCS team has undertaken an initiative: the development of MotokoPilot.

Aiming to bridge the gap left by existing LLMs, the MotokoPilot project involves the training of state-of-the-art AI models using a comprehensive dataset derived from various open-source Motoko code repositories. By fine-tuning these base models to cater specifically to Motoko, we are attempting to deliver a robust and intelligent development solution tailored specifically for Internet Computer development.

In line with this goal, we are also developing a VS Code extension that seamlessly integrates our trained models. This integration is designed to address various aspects of the development process, including code generation, documentation generation, and debugging or refactoring code.

Dataset Creation and Model Training

The creation of our dataset involved a semi-automated process of scraping Motoko code files from open-source repositories, the Dfinity Developer Forum, as well as other publicly available sources. We parsed these files into smaller code fragments using custom-built tooling. These fragments were then used as prompts for generating natural language descriptions, supervised by a researcher, using the GPT-3.5 API.

The result? A comprehensive and highly detailed dataset of prompt-completion pairs that forms the bedrock of our fine-tuned Motoko model, created using OpenAI libraries.

VS Code Extension Development

Our VS Code extension aims to provide a comprehensive solution to developers working with the Motoko language. We have also placed great care in ensuring the extension’s compatibility with other popular tools, such as GitHub Copilot. With a simple but effective user interface, users can access AI-powered features through customizable hotkeys or the right click context menu.

Dataset Enhancement

A key part of this strategy is a thorough review and sanitization of our original dataset. We plan to eliminate any irrelevant or low-quality code samples and ensure that the dataset is representative of high-quality Motoko code. This meticulous curation will enable our AI models to generate code suggestions that are more accurate and relevant to developers’ needs. After we’ve established a solid foundation and tested our prompt-completion pair generation method, the dataset will be incrementally updated with more sources and repositories, further enhancing the final product.

Leveraging Advanced Base Models

In anticipation of access to the GPT-4 API, we aim to leverage its advanced capabilities to create an enhanced dataset of prompt-completion pairs. The GPT-4 API shows significant improvements in both code and natural language understanding and generation capabilities and integrating it into our methodology is expected to substantially enhance the quality of the AI-generated code and documentation.

Harnessing Human Feedback

To fine-tune the model, we intend to implement a modified Reinforcement Learning from Human Feedback (RLHF) approach. This involves engaging community volunteers to evaluate the quality and correctness of the AI-generated output and providing feedback. This iterative refinement process will align the AI-generated output with the expectations of developers.

Once these improvements are in place, we plan to conduct a comprehensive user study with a broad spectrum of developers. The study aims to evaluate the effectiveness of the MotokoPilot extension, focusing on performance, completion time, and overall user experience. The invaluable feedback obtained from this study will fuel further enhancements to our extension. We welcome all developers, regardless of experience, to participate in this beta test. Submissions are already open on our website.

Here at ICPCS, we are deeply committed to ensuring that MotokoPilot is not just a tool but a valuable companion for the Motoko development community. By reinforcing developer productivity and accelerating the learning process, we aspire to contribute substantially to the success of Motoko and the Internet Computer ecosystem at large.

As we navigate this exciting journey, we are optimistic about the transformative potential that MotokoPilot holds for IC developers as well as the entire ICPCS community. Stay tuned for more updates as we forge ahead in our mission!

- — -

Sign up for the Motokopilot Beta
MotokoPilot Technical Report (WandB)

Email: hello@icpcs.io

Originally published at https://icpcs.medium.com on June 22, 2023.

MotokoPilot: Harnessing AI to Empower the Motoko Development Community was originally published in The Internet Computer Review on Medium, where people are continuing the conversation by highlighting and responding to this story.