menu search
brightness_auto
more_vert
0 2

Samiksha, a student of class X was exploring the Natural Language Processing
domain. She got stuck while performing the text normalization. Help her to
normalize the text on the segmented sentences given below:
Document 1: Akash and Ajay are best friends.
Document 2: Akash likes to play football but Ajay prefers to play online games.

Topic Natural Language Processing 
Type Sample Question 2022-2023 CBSE
Class 10

thumb_up_off_alt 0 like thumb_down_off_alt 0 dislike
more_vert

Text normalization involves transforming text into a standard(base) form that can be easily processed by NLP algorithms. Here are some common methods for text normalization:

  1. Tokenization: Breaking text into individual words or tokens.
  2. bringing the text into common case: Converting all words to lowercase to avoid case sensitivity.
  3. Stop word removal: Removing frequently occurring words that do not carry much meaning, such as "the," "and," and "a."
  4. Stemming or Lemmatization: Reducing inflected or derived words to their base or dictionary form, such as "playing" to "play" or "cars" to "car."

Here's how we can normalize the text in the given documents:

Document 1:

2 Answers

more_vert
 
verified
Verified Answer
0

1. Tokenisation 

  • Akash, and, Ajay, are, best, friends Akash, likes, to, play, football, but, Ajay, prefers, to, play, online, games 

2. Removal of stopwords

  • Akash, Ajay, best, friends Akash, likes, play, football, Ajay, prefers, play, online, games 

3. Converting text to a common case 

  • akash, ajay, best, friends akash, likes, play, football, ajay, prefers, play, online, games 

4. Stemming/Lemmatisation 

  • akash, ajay, best, friend akash, like, play, football, ajay, prefer, play, online, game 

(1 mark for each step; 1*4=4)



Learn more about Bag of Words at NLP NLP Processing Class 10

🔥 NATURAL LANGUAGE PROCESSING Class 10 in 60 Mins | ONE SHOT Code 417 | CBSE 2023

thumb_up_off_alt 0 like thumb_down_off_alt 0 dislike
more_vert
0

TEXT NORMALIZATION 

TOKNEISATION : Akash, and, Ajay, are, best, friends, Akash, likes, to, play, football, but, Ajay, prefers, to, play, online ,games

REMOVAL OF STOPWORDS 

Akash, Ajay, best, friends, Akash, likes, play, football, Ajay, prefers, play, online, games

Converting text to common case

akash, ajay, best,friends,akash,likes,,play,football,ajay,prefers,online,games
STEMMING 

akash, ajay,best,friend,akash,like,play,football,ajay,prefers,online,games

LEMMATIZATION 

akash, ajay,best,friend,akash,like,play,football,ajay,prefers,online,games

thumb_up_off_alt 0 like thumb_down_off_alt 0 dislike

Related questions

thumb_up_off_alt 1 like thumb_down_off_alt 0 dislike
1 answer
thumb_up_off_alt 2 like thumb_down_off_alt 0 dislike
1 answer
thumb_up_off_alt 0 like thumb_down_off_alt 0 dislike
1 answer
thumb_up_off_alt 0 like thumb_down_off_alt 0 dislike
1 answer
thumb_up_off_alt 0 like thumb_down_off_alt 0 dislike
1 answer
thumb_up_off_alt 3 like thumb_down_off_alt 0 dislike
1 answer
thumb_up_off_alt 2 like thumb_down_off_alt 0 dislike
1 answer
thumb_up_off_alt 2 like thumb_down_off_alt 0 dislike
1 answer
Welcome to Aiforkids, where you can ask questions and receive answers from other members of the community.

Class 10 Complete One Shot AI Lectures at - Youtube

1.5k questions

1.4k answers

4 comments

4.1k users

...