What is LaMDA?

LaMDA (Language Model for Dialogue Applications) is one of the latest language models developed by Google to understand the nuances of language in search queries. This language model is built on Transformer and trained on dialogue. So, it can answer open-ended questions and have a more humanlike understanding of the language. This feature sets LaMDA apart from the rest of the chatbots.

Objectives

LaMDA is built with three key objectives, i.e., groundedness, safety, and quality. Groundedness means that any factual information passed from LaMDA is based on credible external links on the internet. Safety refers to LaMDA being a responsible AI product by not posing any risk of harm to the user and avoiding any bias. Quality is defined in terms of three factors, i.e., sensibility, specificity, and interestingness. This means that the responses of the model should make sense, answer the question asked by the user, and be able to create humanlike dialogues.

Training

LaMDA’s training comprised two stages, i.e., pre-training and fine-tuning. The pre-training dataset comprised dialogue and web documents containing 1.56T words. After pretraining, the model was fine-tuned as a generator and classifier. While the generator generates several responses, the classifier evaluates the responses based on the objectives and selects the best one to proceed with. Both training stages ensured that the model abides by the AI principles of Google.

Limitations

Despite being a powerful language model, LaMDA, too, has some limitations. First, the model’s performance is highly dependent on the data. If there are any biases in the training data, they will be reflected in the model. There is also a possibility that LaMDA is unable to fact-check for something it was not trained on. Second, although the model can understand and conduct conversations, it can still struggle with complex sentences because of its dependency on the context to understand the text. Additionally, LaMDA is currently limited to dialogues only and is not capable of producing code or nontextual responses.

Applications

The model’s evaluation revealed that LaMDA outperformed every other model and humans in the quality objective but struggled with safety and groundedness, which was exceeded by humans. This shows that LaMDA has exceptional abilities as a conversational agent and understanding human queries that will significantly impact the field of SEO.