Question

Through a step-by-step process, calculate TFIDF for the given corpus:

Document 1: To the swinging and the ringing

Document 2: of the bells, bells, bells

Document 3: Of the bells, bells, bells, bells

Document 4: Bells, bells, bells

Document 5: To the rhyming and the chiming of the bells.

Topic	Natural Language Processing (AI Domain)
Type	Short answer type
Class	10

Answer 1

Step 1: Create document vectors for the given documents (Term Frequency Table):-

To	The	Swinging	and	ringing	of	bells	rhyming	chiming
1	1	1	1	1	0	0	0	0
0	1	0	0	0	1	3	0	0
0	1	0	0	0	1	4	0	0
0	0	0	0	0	0	3	0	0
1	1	0	1	0	1	1	1	1

Step 2: Record the occurrence of the word in the document using the term frequency table (Document Frequency Table):-

To	The	swinging	and	ringing	of	bells	rhyming	chiming
2	4	1	2	1	3	11	1	1

Step 3: Draw the inverse document frequency table wherein, we need to put the document frequency in the denominator while the total number of documents is the numerator. Here, the total number of documents is 5, hence inverse document frequency becomes:-

To	The	swinging	and	ringing	of	bells	rhyming	chiming
5/2	5/4	5/1	5/2	5/1	5/3	5/11	5/1	5/1

Step 4: The formula of TFIDF for any word W becomes: TFIDF(W) = TF(W) * log (IDF(W)):-

To	the	swinging	and	ringing	of	bells	rhyming	chiming
1*log(5/2)	1*log(5/4)	1*log(5/1)	1*log(5/2)	1*log(5/1)	0*log(5/3)	0*log(5/11)	0*log(5/1)	0*log(5/1)
0*log(5/2)	1*log(5/4)	0*log(5/1)	0*log(5/2)	0*log(5/1)	1*log(5/3)	3*log(5/11)	0*log(5/1)	0*log(5/1)
0*log(5/2)	1*log(5/4)	0*log(5/1)	0*log(5/2)	0*log(5/1)	1*log(5/3)	4*log(5/11)	0*log(5/1)	0*log(5/1)
0*log(5/2)	0*log(5/4)	0*log(5/1)	0*log(5/2)	0*log(5/1)	0*log(5/3)	3*log(5/11)	0*log(5/1)	0*log(5/1)
1*log(5/2)	1*log(5/4)	0*log(5/1)	1*log(5/2)	0*log(5/1)	1*log(5/3)	1*log(5/11)	0*log(5/1)	0*log(5/1)

---

Study more about Natural Language Processing at Natural Language Processing Class 10