A chatbot is an artificial intelligence powered piece of software in a device, application, web site or alternative networks that try to complete consumer’s needs and then assist them to perform a selected task. Now a days almost every company has a chatbot deployed to interact with the users.

Chatbots are often used in many departments, businesses and every environment. They are artificial narrow intelligence (ANI). Chatbots only do a restricted quantity of task i.e. as per their design. However, these Chatbots make our lives easier and convenient. The trend of Chatbots is growing rapidly between businesses and entrepreneurs, and are willing to bring chatbots to their sites. You might also produce it yourself using Python.

How do chatbots work?

There are broadly two variants of chatbots: Rule-Based and Self learning.

1. In a Rule-based approach, a bot answers questions based on some rules on that it is trained on. The rules outlined could be very easy to very complicated. The bots will handle easy queries but fail to manage complicated ones.
2. The Self learning bots are those that use some Machine Learning-based approaches and are positively a lot of economical than rule-based bots. These bots may be of additional two types: Retrieval based or Generative.
   1. In retrieval-based models, Chatbot uses the message and context of conversation for selecting the best response from a predefined list of bot messages.
   2. Generative bots can generate the answers and not always reply with one of the answers from a set of answers. This makes them more intelligent as they take word by word from the query and generates the answers.

Building a chatbot using Python

NLP:

The field of study that focuses on the interactions between human language and computers is called Natural Language Processing. NLP is a way for computers to analyze, understand, and derive meaning from human language in a smart and useful way. However, if you are new to NLP, you can read Natural Language Processing in Python.

NLTK:

NLTK (Natural Language Toolkit) is a leading platform for building Python programs to work with human language data. It provides easy-to-use lexical resources such as WordNet, along with a suite of text processing libraries.

Importing necessary libraries

import nltk 

import numpy as np 

import random 

import string # to process standard python strings

Copy the content in text file named ‘chatbot.txt’, read in the text file and convert the entire file content into a list of sentences and a list of words for further pre-processing.

f=open('chatbot.txt','r',errors = 'ignore')

raw=f.read()

raw=raw.lower()# converts to lowercase

nltk.download('punkt') # first-time use only

nltk.download('wordnet') # first-time use only

sent_tokens = nltk.sent_tokenize(raw)# converts to list of sentences 

word_tokens = nltk.word_tokenize(raw)# converts to list of words

Pre-processing the raw text

We shall now define a function called LemTokens which will take as input the tokens and return normalized tokens.

lemmer = nltk.stem.WordNetLemmatizer()

#WordNet is a semantically-oriented dictionary of English included in NLTK.

def LemTokens(tokens):     

return [lemmer.lemmatize(token) for token in tokens]

remove_punct_dict = dict((ord(punct), None) for punct in string.punctuation) 

def LemNormalize(text):     

return LemTokens(nltk.word_tokenize(text.lower().translate(remove_punct_dict)))

Keyword matching

Define a function for greeting by bot i.e. if user’s input is greeting, the bot shall return a greeting response.

GREETING_INPUTS = ("hello", "hi", "greetings", "sup", "what's up","hey",)

GREETING_RESPONSES = ["hi", "hey", "*nods*", "hi there", "hello", "I am glad! You are talking to me"]

def greeting(sentence):

for word in sentence.split():

if word.lower() in GREETING_INPUTS:

return random.choice(GREETING_RESPONSES)

Generate responses

To generate a response from our bot for input queries, the concept of document similarity is used. Therefore, we start by importing necessary modules.

From scikit learn library, import the TFidf vector to convert a collection of raw documents to a matrix of TF-IDF features

from sklearn.feature_extraction.text import TfidfVectorizer

Also, import cosine similarity module from scikit learn library

from sklearn.metrics.pairwise import cosine_similarity

This will be used to find the similarity between words entered by the user and therefore the words within the corpus. This can be the simplest possible implementation of a chatbot.

Define a function response that searches the user’s vocalization for one or more known keywords and returns one of several possible responses. If it doesn’t find the input matching any of the keywords, it returns a response: “I’m sorry! I don’t understand you”

def response(user_response):

robo_response=''

sent_tokens.append(user_response)

TfidfVec = TfidfVectorizer(tokenizer=LemNormalize, stop_words='english')

tfidf = TfidfVec.fit_transform(sent_tokens)

vals = cosine_similarity(tfidf[-1], tfidf)

idx=vals.argsort()[0][-2]

flat = vals.flatten()

flat.sort()

req_tfidf = flat[-2]

if(req_tfidf==0):

robo_response=robo_response+"I am sorry! I don't understand you"

return robo_response

else:  robo_response = robo_response+sent_tokens[idx]

return robo_response

I have tried to explain in simple steps how you can build your own chatbot using NLTK and of course it’s not an intelligent one.

I hope you guys have enjoyed reading.

Happy Learning!!!

NLP is a branch of data science that consists of systematic processes for analyzing, understanding, and deriving information from the text information in a smart and efficient manner. By utilizing NLP and its parts, one can organize the massive chunks of text information, perform various automated tasks and solve a wide range of issues like – automatic summarization, machine translation, named entity recognition, relationship extraction, sentiment analysis, speech recognition, and topic segmentation etc.

NLTK (Natural Language Toolkit) is a leading platform for building Python programs to work with human language data. It provides easy-to-use interfaces to lexical resources like WordNet, along with a collection of text processing libraries for classification, tokenization, stemming, and tagging, parsing, and semantic reasoning, wrappers for industrial-strength NLP libraries.

NLTK has been called “a wonderful tool for teaching and working in, computational linguistics using Python,” and “an amazing library to play with natural language.”

Downloading and installing NLTK

1. Install NLTK: run pip install nltk
2. Test installation: run python then type import nltk and run nltk.download() and download all packages.

Pre-Processing with NLTK

The main issue with text data is that it's all in text format. However, the Machine learning algorithms need some variety of numerical feature vector so as to perform the task. Thus before we have a tendency to begin with any NLP project we'd like to pre-process it to form it ideal for working. Basic text pre-processing includes:

• Converting the whole text into uppercase or lowercase, in order that the algorithm doesn't treat the same words completely different in several cases.
• Tokenization: Process of converting the normal text strings into a list of tokens i.e. words that we actually want. The NLTK data package includes a pre-trained Punkt tokenizer for English.

           import nltk

           from nltk.tokenize import word_tokenize

           text = "God is Great! I won a lottery."

           print(word_tokenize(text))

           Output: ['God', 'is', 'Great', '!', 'I', 'won', 'a', 'lottery', '.']

• Noise removal: Process of removing everything that isn’t in a standard number or letter.
• Stop word removal: A stop word is a commonly used word (such as “the”, “a”, “an”, “in”). We would not want these words or taking up valuable processing time. For this, we can remove them easily, by storing a list of words that you consider to be stop words. NLTK (Natural Language Toolkit) in python has a list of stopwords stored in sixteen different languages. You can find them in the nltk_data directory.  home/Saad/nltk_data/corpora/stopwords is the directory address.

           import nltk

           from nltk.corpus import stopwords

           set(stopwords.words('english'))

• Stemming: Stemming is the process of reducing the words to its root form. Example if we were to stem the following words: “Connects”, “Connecting”, “Connected”, “and Connection”, the result would be a single word “Connect”.

           # import these modules

           from nltk.stem import PorterStemmer

           from nltk.tokenize import word_tokenize   

           ps = PorterStemmer()  

           # choose some words to be stemmed

           words = ["Connect", "Connects", “Connected”, "Connecting", "Connection", "Connections"]

           for w in words:

           print(w, " : ", ps.stem(w)) 

• Lemmatization: Lemmatization is the process of grouping along the various inflected forms of a word in order that they may be analyzed as a single item. Lemmatization is similar to stemming but it brings context to the words. Therefore it links words with similar meaning to one word.

           # import these modules

           from nltk.stem import WordNetLemmatizer  

           lemmatizer = WordNetLemmatizer()  

           print("rocks :", lemmatizer.lemmatize("rocks"))

           print("corpora :", lemmatizer.lemmatize("corpora"))  

           # a denotes adjective in "pos"

          print("better :", lemmatizer.lemmatize("better", pos ="a"))

          -> rocks : rock

          -> corpora : corpus

          -> better : good

Now we need to transform text into a meaningful vector array. This vector array is a representation of text that describes the occurrence of words within a document. For example, if our dictionary contains the words {Learning, is, the, not, great}, and we want to vectorize the text “Learning is great”, we would have the following vector: (1, 1, 0, 0, 1). A problem is that extremely frequent words begin to dominate within the document (e.g. larger score), however might not contain as much informational content. Also, it will offer additional weight to longer documents than shorter documents.

One approach is to rescale the frequency of words or the scores for frequent words called Term Frequency-Inverse Document Frequency.

• Term Frequency: is a scoring of the frequency of the word in the current document.

           TF = (Number of times term t appears in a document)/ (Number of terms in the document)

• Inverse Document Frequency: It is a scoring of how rare the word is across documents.

           IDF = 1+log(N/n), where, N is the number of documents and n is the number of documents a term t has appeared in.

           Tf-idf weight is a weight often used in information retrieval and text mining.

           Tf-IDF can be implemented in scikit learn as:

           from sklearn.feature_extraction.text import TfidfVectorizer

           corpus = [

           ...     'This is the first document.’

           ...     'This document is the second document.’

           ...     'And this is the third one.’

           ...     'Is this the first document?',]

           >>> vectorizer = TfidfVectorizer()

           >>> X = vectorizer.fit_transform(corpus)

           >>> print(vectorizer.get_feature_names())

           ['and', 'document', 'first', 'is', 'one', 'second', 'the', 'third', 'this']

           >>> print(X.shape)

           (4, 9)

• Cosine similarity: TF-IDF is a transformation applied to texts to get two real-valued vectors in vector space. We can then obtain the Cosine similarity of any pair of vectors by taking their dot product and dividing that by the product of their norms. That yields the cosine of the angle between the vectors. Cosine similarity is a measure of similarity between two non-zero vectors.

           Cosine Similarity (d1, d2) =  Dot product(d1, d2) / ||d1|| * ||d2||

          import numpy as np

          from sklearn.metrics.pairwise import cosine_similarity

          # vectors

          a = np.array([1,2,3])

          b = np.array([1,1,4])

          # manually compute cosine similarity

          dot = np.dot(a, b)

          norma = np.linalg.norm(a)

          normb = np.linalg.norm(b)

          cos = dot / (norma * normb)

After completion of cosine similarity matric we perform algorithmic operation on it for Document similarity calculation, sentiment analysis, topic segmentation etc.

I have done my best to make the article simple and interesting for you, hope you found it useful and interesting too.

Over the last year or so, programming languages have regularly been prefixed with a curious word: modern, Modern Java. But what exactly does modern mean when used in this way?

When someone talks about modern languages, they're really just talking about how refined, how advanced and how convenient a programming language is. This also means that the language is capable enough not just to solve problems of the present, but of the future as well. A long line of features like scalability, being cloud-ready, and supporting newer paradigms and architectures is expected of a "modern" programming language.

Today, in fact, java is the most used runtime platform on enterprise systems (more than 97% of desktops). But more than that, its virtual machine powers packages and custom business applications, and a wide array of mobile and other embedded platforms.

Currently, according to Oracle, more than 3 billion devices run Java in some form. Most major companies use Java for some of their functions and Java server applications are processing tens of millions of requests each day.

Why java is so popular?

One of the most important reasons why Java is so popular is the platform independence. Java is a concurrent, class-based, and object-oriented programming language. It was initially designed to have as few implementation dependencies as possible, which lead to the term "write once, run anywhere" (WORA). This means that compiled Java code can run on all platforms with no need for recompiling the code.

Java-based applications are known for their speed and scalability. Its efficient processing speeds are used in software, computer games, and mobile Apps. Java supports Multithreading. Multithreading means handling more than one job at a time, so get more process get done in less time than it could with just one thread. Java is also a statically typed language, so that it brings a much greater degree of safety and stability to its programs compared to other popular languages. This safety and stability is a necessity for companies who require major bandwidth in their software and apps.

Is Java worth learning?

Java is still a relevant programming language that shows no sign of declining in popularity. Most developers choose it up as their initial programming language because it's reasonably simple to learn.

Since the language has an English-like syntax with minimum special characters, Java could be learned in a very short time span and used to build appropriate applications. It is part of a family of languages that are heavily influenced by C++ (as well as C#), thus learning Java offers vast benefits when learning these alternative two languages.

"Developing programs is a kind of making art, once you learn clearly and spend your time with full involvement; the creation of art became so easy and simple."

General Advantages of Using Java for Business Applications:

Programming with Java is incredibly common for banking and web applications. Compared to other programming languages, Java definitely stands out in terms of security functionality and environment. It comes with certain built-in security features such as:

Java apps are able to manage their own use by multiple users at the same time, creating threads for each use within the program itself, rather than having to run multiple copies of the programming in the same hardware. Each thread is tracked until the "work" is finished.

Java is so versatile and provides robust customized solutions for almost any type of business need. This "referred position" shows no signs of declining, especially now that Java 10 is on the horizon. It just keeps getting better.

Advanced Authentication and Access Control that allows incorporating a range of secure login mechanisms, along with creating the custom security policy and enforce a well-defined permission access policy to sensitive data.

Cryptography

Advanced Authentication and Access Control that allows incorporating a range of secure login mechanisms, along with creating the custom security policy and enforce a well-defined permission access policy to sensitive data.

Java apps are able to manage their own use by multiple users at the same time, creating threads for each use within the program itself, rather than having to run multiple copies of the programming in the same hardware. Each thread is tracked until the "work" is finished.

Java is so versatile and provides robust customized solutions for almost any type of business need. This "referred position" shows no signs of declining, especially now that Java 10 is on the horizon. It just keeps getting better.