source: Nick Youngson CC BY-SA 3.0 ImageCreator

Question Analysis

Question Analysis for Question Answering System.

In a question answering system, it is important to know the type of question so as to get the answer. For example, “who is Pema?” we can obviously say that it's asking about a person. So how can we make the machine understand that too?

As said in the paper “Information Retrieval: Improving Question Answering Systems by Query Reformulation and Answer Validation” by Mohammad Reza Kangavari, Samira Ghandchi, Manak Golpour[1], there are two important components which are the bases of the question processing. The first component is the question classification that specifies types of question and answer.

Table 1 Classification of question and answer

Referring to the paper and also applying some other methods, what I did was firstly get NER using NLTK:

Fig: question analysis using NLTK library

We can automatically tell that the question is about “PERSON”. But if the question is “where should I come?” NLTK library returns null, for such cases, I applied the rules from the paper. The output generated:

Fig: question analysis referring to the paper

code for the above output: https://github.com/pemagrg1/Question-Analysis

For which, I have used:

1. Stemming
2. Chunkgrams
3. POS tagging
4. NER
5. Question Taggers
6. Rules

Reference:
[1] http://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.193.1603&rep=rep1&type=pdf