• <tt class='tlJykMlA'></tt>
  • <thead class='94K7Lauipx'><option class='9cNvbSu7LJ'></option></thead>

    <em class='jepytdnyfeW3'><b class='oR1Ys12nk'><td class='6rzzLTN'></td></b></em>

  • <dl class='Td0EmkF'><b class='qoWhaibfJ9'></b></dl>

  • <span class='lS1G'></span>

     

    KDnuggets 500彩票下载app二维码 » News » 2020 » Jan » Tutorials, Overviews » Generating English Pronoun Questions Using Neural Coreference Resolution ( 20:n05 )

    Generating English Pronoun Questions Using Neural Coreference Resolution


     
     

    This post will introduce a practical method for generating English pronoun questions from any story or article. Learn how to take an additional step toward computationally understanding language.



    By , NLP, AI Freelancer

    Figure

    Neural Coreference resolution

     

    In this post we will see how to generate English pronoun questions from any story or article. This is one step towards automatically generating English language learning worksheet.

    The input to our program will be a small story like the following -

    Scientists know many things about the Sun. They know how old it is. The Sun is more than 4 1/2 billion years old. It is also a star that is the centre of our solar system. They also know the Sun’s size.

    The output from the program will be a set of pronoun questions -

    1. What does “They” refer to in the sentence — “They know how old it is.”? Ans : Scientists
    2. What does “They” refer to in the sentence — “They also know the Sun’s size.”? Ans : Scientists
    3. What does “It” refer to in the sentence — “It is also a star that is the centre of our solar system.”? Ans : The sun

    Let’s get started to see how we can achieve this using the from hugging face.

    All the code and the jupyter notebook is available at -


    Use huggingface neuralcoref library to automatically generate english pronoun questions from any story or article.
     

    First install the necessary libraries in the jupyter notebook :

    !pip install neuralcoref
    !pip install spacy==2.1.0
    !python3 -m spacy download en


    Initialize neural coreference library:

    # Load you500彩票下载app二维码r usual SpaCy model (one of SpaCy English models)
    import spacy
    nlp = spacy.load('en')
    import re
    from nltk.tokenize import sent_tokenize# Add neural coref to SpaCy's pipe
    import neuralcoref
    neuralcoref.add_to_pipe(nlp,greedyness=0.5,max_dist=50,blacklist=False)


    Initialize with some sample text and resolve coreferences with the library:

    text = "Scientists know many things about the Sun. They know how old it is. The Sun is more than 4 1/2  billion years old. It is also a star that is the centre of our solar system. They also know the Sun’s size."text = str(text)doc = nlp(text)clusters = doc._.coref_clustersprint("clusters ",clusters)
    print ("\n\n")
    resolved_coref = doc._.coref_resolved
    print ("Resolved by NeuralCoref: \n" )
    print(resolved_coref)


    The print output from clusters will be:

    [Scientists: [Scientists, They, They], the Sun: [the Sun, The Sun, Sun, It]]

    For each entity you500彩票下载app二维码 will see the coreferences of that entity and also any pronouns associated with it.

    When you500彩票下载app二维码 let the library resolve coreferences (replace pronouns with their nouns) the coreference resolved output is :

    Scientists know many things about the Sun. Scientists know how old it is. the Sun the Sun is more than 4 1/2  billion years old. the Sun is also a star that is the centre of our solar system. Scientists also know the Sun’s size.


    Because of resolving errors as well as perhaps some indexing errors in the library logic the ouput is some times odd. Eg: “the sun the sun” in second line.

    We will attempt to write our custom resolving function as well as generate english pronoun grammar questions in the process.

    First we choose only a subset of pronouns to be replaced. And write an auxiliary function that is helpful for us to get a sentence given an index of a word.

    Then we write our custom function to resolve only those coreferences from the cluster list that are pronouns and are not the same as the original entities.

    With our custom coreference function above, the output for the initial text with coreference resolved is —

    Scientists know many things about the Sun. Scientists know how old it is. The Sun is more than 4 1/2 billion years old. The sun is also a star that is the centre of our solar system. Scientists also know the Sun’s size.

    And finally printing out the questions -

    print ("\nQuestions generated :")
    print ("[Note: There might be a few answer errors because of the errors in the coreference algorithm itself] \n")for index,question in enumerate(questions):
        print ('%d) What does \"%s\" refer to in the sentence - \"%s\"?'%(index+1,question[1],question[0].strip()))
        print ("Ans : %s\n"%(question[2]))


    The output is :

    Questions generated :
    [Note: There might be a few answer errors because of the errors in the coreference algorithm itself]
    
    1) What does “They” refer to in the sentence — “They know how old it is.”?
    Ans : Scientists
    
    2) What does “They” refer to in the sentence — “They also know the Sun’s size.”?
    Ans : Scientists
    
    3) What does “It” refer to in the sentence — “It is also a star that is the centre of our solar system.”?
    Ans : The sun
    

     

    Happy coding ! For any questions or just to say hi reach out to me at ramsrigouthamg@500彩票下载app二维码gmail.com

     
    Bio: is an NLP and AI Freelancer. He is currently building an AI-Assisted tool for educators to make the process of creating assessments faster and better.

    . Reposted with permission.

    Related:


    Sign Up

    By subscribing you500彩票下载app二维码 accept KDnuggets Privacy Policy

    500彩票下载app二维码

  • <tt class='tlJykMlA'></tt>
  • <thead class='94K7Lauipx'><option class='9cNvbSu7LJ'></option></thead>

    <em class='jepytdnyfeW3'><b class='oR1Ys12nk'><td class='6rzzLTN'></td></b></em>

  • <dl class='Td0EmkF'><b class='qoWhaibfJ9'></b></dl>

  • <span class='lS1G'></span>