menu search
brightness_auto
more_vert
1 1

Through a step-by-step process, calculate TFIDF for the given corpus and mention the word(s) having the highest value

Document 1: We are going to Mumbai

Document 2: Mumbai is a famous place.

Document 3: We are going to a famous place. 

Document 4: I am famous in Mumbai.

Topic   Natural Language Processing (AI Domain)
Type  Long answer type
Class 10

thumb_up_off_alt 1 like thumb_down_off_alt 0 dislike

1 Answer

more_vert
 
verified
Verified Answer
1

Term Frequency:-

Term frequency is the frequency of a word in one document. Term frequency can easily be found from the document vector table as in that table we mention the frequency of each word of the vocabulary in each document.

We  Are  Going To   Mumbai
Is   

a   
famous place
I    
am  in  
1 1 1 1 1 0 0 0 0 0 0 0
0 0 0 0 1 1 1 1 1 0 0 0
1 1 1 1 0 0 1 1 1 0 0 0
0 0 0 0 1 0 0 1 0 1 1 1

Inverse Document Frequency:- 

 
The other half of TFIDF which is Inverse Document Frequency. For this, let us first understand what does document frequency mean. 
Document Frequency is the number of documents in which the word occurs irrespective of how many times it has occurred in those documents. The document frequency for the exemplar vocabulary would be:
We  Are  Going To   Mumbai Is    a    famous place I     am  in  
2 2 2 2 3 1 2 3 2 1 1 1

 

Talking about inverse document frequency, we need to put the document frequency in the denominator while the total number of documents is the numerator. Here, the total number of documents are 4, hence inverse document frequency becomes:

We  Are  Going To   Mumbai Is    a    famous place I     am  in  
4/2 4/2 4/2 4/2 4/3 4/1 4/2 4/3 4/2 4/1 4/1 4/1

The formula of TFIDF for any word W becomes:
TFIDF(W) = TF(W) * log (IDF(W)
The words having highest value are – Mumbai, Famous 

Study more about Natural Language Processing at Natural Language Processing Class 10

thumb_up_off_alt 1 like thumb_down_off_alt 0 dislike

Related questions

thumb_up_off_alt 2 like thumb_down_off_alt 0 dislike
1 answer
thumb_up_off_alt 1 like thumb_down_off_alt 0 dislike
1 answer
thumb_up_off_alt 3 like thumb_down_off_alt 0 dislike
1 answer
thumb_up_off_alt 1 like thumb_down_off_alt 0 dislike
1 answer
thumb_up_off_alt 1 like thumb_down_off_alt 0 dislike
1 answer
thumb_up_off_alt 1 like thumb_down_off_alt 0 dislike
1 answer
thumb_up_off_alt 2 like thumb_down_off_alt 0 dislike
1 answer
thumb_up_off_alt 1 like thumb_down_off_alt 0 dislike
1 answer
Welcome to Aiforkids, where you can ask questions and receive answers from other members of the community.

AI 2024 Class 10 Board Exams mein 100% laane ka plan OPEN NOW

Class 10 Complete One Shot AI Lectures at - Youtube

1.5k questions

1.4k answers

4 comments

5.0k users

...