Building our own Chatbot and Cloud AI Service

Welcome to Chatty Coffee, a chatbot AI experiment with NLP.js and Netlify functions.

Natural Language Processing, or NLP for short, is a part of artificial intelligence (AI) that deals with the communication between machines and humans using natural language or ordinary language.

The ultimate purpose of NLP is to parse, understand, and make sense of the human language in a structured manner.

With this technology, you can, for example, build advanced and sophisticated chatbots. Of course, there are existing services like Amazon Lex and DialogFlow from Google, but they share your data with those companies. Using NLP.js, you can bootstrap your own private chatbot and cloud AI service. In this blog post, I will explain how.

Let’s order some Chatty Coffee!

The hardest part of building chatbots is to understand what our end-users want. In other words, what are their end-goals? Once we know that, we can present them with relevant data. In the NLP world, we call these goals: intents.

Let's say we are building a chatbot for a new, fancy coffee shop. The most important feature our bot should have is the ability to understand and place user orders. So, our first intent is order.

There are infinite ways our users can phrase the same intent, for example, they could say: "Can I have one triple espresso please?" but they could also say: "I want one triple espresso.". Thankfully, this is not a challenge for NLP systems, no matter how users phrase themselves, their intent is the same. All we need to do is to give the system a few training examples, and it will figure the rest by itself.

const { NlpManager } = require("node-nlp");
const manager = new NlpManager({ languages: ["en"], threshold: 0.90 });

// I suggest you train each intent with about 10 input examples.
manager.addDocument("en", "Can I get a triple espresso please?", "Order");
manager.addDocument("en", "Can I order a triple espresso please?", "Order");
manager.addDocument("en", "Give me a triple espresso.", "Order");
manager.addDocument("en", "I want a triple espresso.", "Order");
manager.addDocument("en", "One triple espresso please.", "Order");
// etc

(async() => {
  await manager.train();
  manager.save();

  // Once you have a model trained, you can reuse it with:
  // manager.load();

  const userInput = "Hi, I'd like one triple espresso, please.";
  const response = await manager.process(userInput);
  console.log(response);
})();

Result:

{
   "utterance": "Hi, I'd like one triple espresso, please.",
   "nluAnswer": { "classifications": [{ "intent": "Order", "score": 1 }] },
   "intent": "Order",
   "score": 1,
   ...
}

As you can see, our NLP model had no issue figuring out the user’s intent. You can now add as many intents as your use-case requires.

Understanding the user’s order

Now you may be thinking, a triple espresso is nice and all, but how about, lattes and cappuccinos? Good question! Our NLP model is now smart enough to understand that the user wants to place an order. But it does not know which product, what size or how many the user is asking for. It's time to learn about named entities.

A named entity is a piece of relevant information extracted from a user’s intent. For example, the string "Hi, I'd like one triple espresso, please." contains the following entities: ["one", "triple", "espresso"]. They represent all of the needed information to place a new order, the amount, the size, and the "what".

NLP.js comes with build-in entities which are great, but we also need to add custom ones. You can create custom-named entities using addNamedEntityText(ENTITY_TYPE, ENTITY_NAME, LANGUAGES, ALIASES):

manager.addNamedEntityText("size", "grande", ["en"], ["Grande", "Large", "Triple"]);
manager.addNamedEntityText("size", "short", ["en"], ["Short", "Small", "Single"]);
manager.addNamedEntityText("size", "tall", ["en"], ["Tall", "Medium", "Double"]);

Let’s also create named entities for our beverages:

manager.addNamedEntityText("drink", "americano", ["en"], ["Americano", "Americanos"]);
manager.addNamedEntityText("drink", "latte", ["en"], ["Latte", "Lattes"]);
manager.addNamedEntityText("drink", "cappuccino", ["en"], ["Cappuccino", "Cappuccinos"]);
...

Lastly, we can use our new entities inside our existing intents, like this:

manager.addDocument("en", "Can I get a %size% %drink% please?", "Order");
manager.addDocument("en", "Can I order a %size% %drink% please?", "Order");
manager.addDocument("en", "Give me a %size% %drink%.", "Order");
manager.addDocument("en", "I want a %size% %drink%.", "Order");
manager.addDocument("en", "One %size% %drink% please.", "Order");
...

If we now process the same utterance as before (manager.process("Hi, I'd like one triple espresso, please."), we get the following result:

{
  "entities": [
    {
      "accuracy": 0.84,
      "entity": "size",
      "option": "grande",
      "sourceText": "Triple",
      "utteranceText": "tripple"
    },
    {
      "accuracy": 1,
      "entity": "drink",
      "option": "espresso",
      "sourceText": "Espresso",
      "utteranceText": "espresso"
    },
    {
      "accuracy": 0.95,
      "sourceText": "one",
      "utteranceText": "one",
      "entity": "number",
      "resolution": {
        "strValue": "1",
        "value": 1,
        "subtype": "integer"
      }
    }
  ],
  ...
}

Dutch? French? German? No problem!

The NLP.js package also supports multi-language systems. You can easily add additional languages with the following changes:

1. First, we need to update the language array of our NLP manager.

const manager = new NlpManager({ languages: ["en", "nl", "fr", "de"], threshold: 0.90 });

2. Next, we need to expand our training data to accommodate the new languages.

manager.addDocument("nl", "Mag ik een %size% %drink%?", "Order"); // Dutch
manager.addDocument("fr", "Puis-je avoir un %size% %drink%?", "Order"); // French
manager.addDocument("de", "Kann ich einen %size% %drink%?", "Order"); // German
// etc

3. Finally, we need to update the language array for each entity.

// Shared aliases.
manager.addNamedEntityText("size", "grande", ["en", "nl", "fr", "de"], ["Grande", "Large", "Triple"]);

// Language specific.
manager.addNamedEntityText("size", "grande", ["nl"], ["Groot", "Grote"]); // Dutch
manager.addNamedEntityText("size", "grande", ["fr"], ["Grand"]); // French
manager.addNamedEntityText("size", "grande", ["de"], ["Groß", "Großer"]); // German

We can either let the system detect the language automatically or we can pass in the language as a parameter: manager.process("nl", "Mag ik een groot cappuccino?").

That’s it! You now have a fully functional and private multi-language NLP system!