Hello, if you have any need, please feel free to consult us, this is my wechat: wx91due

Assignment1

1 Informatics 

1 Cognitive Science 2024/25
2 Assignment 1

2.0.1 How to use this notebook

Text answers: For questions with a text answer, there will be a markdown cell following thequestion. There will usually be an indication that the cell is intended for your answer such as “YOUR ANSWER HERE”, which you can replace with your response.

Assert statements: You may notice that some lines of code starting with assert cause errors whenrun multiple times. If your answer is correct, these tests should pass when you restart your notebookand run all cells in order.

2.0.2 Submitting your work

You should run your notebook and save your work before you submit.

Once you are ready, go to the assignments tab on the Noteable landing page and click “Submit” on the relevant assignment.

You are allowed to submit multiple times. Please be aware that we will mark your lastsubmission even if this is after the deadline. Therefore to avoid late penalties, you should notsubmit after the deadline. It may take a few minutes to submit your assignment so please bear thisin mind when timing your submission. If you have not previously submitted an assignment through

Noteable, we recommend you submit an early version of your assignment so that you can ensureyou understand this process. You can also walk through this process with a lab demonstrator ifyou are unsure.

Note: The Noteable system has undergone a substantial update this year. As such, there may be changes which we have not yet seen. If you have issues with using the assignment, please let thecourse teaching assistants know so that we can provide support.

If you have any issues submitting: Download your notebook by right-clicking on the file name inthe file tree on the left and selecting “Download”. You can then email this file to the course staff as evidence of a timely submission. You may still need to submit via Noteable so please pay closeattention to any updated instructions from course staff.

3 PART 1

Assignment 1 directory and may be a handy reference. We’re going to walk through the necessary components here.

Most of the functions we need for the RM model are already implemented for us: Have a look at the file RM1985.py if you want to know the details.

3.1 Encoding the input

Words (e.g., came) are translated into phonemes (e.g., kAm).

Phonemes are converted into a binary code based on their articulatory features (see below). Remember that in RM’s model we use wickelphones (triplets of phonemes; e.g., #kA, kAm, Am#).

In practice, we also add a bit representing if a wickelphone touches a word boundary, resulting inan 11 bit code for each phoneme. You will not need the specifics of the code for this assignment.

Wickelphones are converted into wickelfeatures by taking one of the bits from the first phoneme in awickelphone, one of the bits from the second phoneme, and one of the bits from the third. In theorythen, there are 11 * 11 * 11 possible wickelfeatures. However, in practice a lot of these combinationsare redundant and we can represent all informative combinations with just 460 wickelfeatures. Ifyou’re really itching for the details they’re in the paper.

Luckily, we have a helper function that will do all of this encoding for us: activate_word

We should check some properties of our encoding function.

A lot of letters in English map directly onto their phoneme (e.g., b -> b) however, there are a few that don’t (e.g., c as in car is represented by a k). The table below should help you translate wordsinto phonemes (particularly the caption helps with vowels):

Exercise 1a)

Now let’s check some properties of our encoding function: activate_word

As mentioned above, there should be 460 wickelfeatures for each word. We can check this using np.shape

For a three phoneme word like came, we expect there to be at most 16 active wickelfeatures.

Store the shape of the wickelfeature represetation for came and the number of wickelfeatures acti vated (i.e., set to 1). (Points 2)

## The Model

Now that we have our input representations, we can work on the model implementation.

The goal of RM’s model is to output a wickelfeature representation for the past tense of the input verb. To do this, they build a perceptron for each of the 460 possible wickelfeatures. Each perceptron receieves the entire phonological representation of the base verb as input and decides ifa single wickelfeature should be present in the output.

In the standard perceptron, a unit fires if the weighted activation net is greater than a threshold theta; otherwise it does not fire.

In RM’s perceptron, a unit fires probabilistically according to the activation function:

Hint: the exponential function can be calculated using np.exp()

RM’s activation function looks quite similar to the sigmoid function we discussed in lecture 6.

However, what does the T parameter do?

Let’s set theta = 0.0 and plot the probability of firing as a function of the the weighted activation net at T=1.0, T=0.5 and T=2.0 in one figure. We should consider weighted activation values between -5 and 5.

Please have a look at lab 3 if you need a reminder on how to do plotting in Python!

YOUR ANSWER HERE

4 U Shaped Curves

First they trained the model on ten high frequency words (8 irregular, 2 regular), then medium frequency verbs, then low frequency verbs. They then plotted the percent of wickelfeatures themodel correctly predicted for irregular and regular verbs as the model saw more data.

Let’s replicate this step by step :)

First, let’s load in a corpus of English verbs.

In the first stage, RM trained the model on the high frequency verbs.

First we need to convert the phonemes into wickelfeatures.

Next we initialize a perceptron with our custom activation function

The function Perceptron.learn() does one epoch of weight updates and then returns the model accuracy on the data. This is the percentage of the wickelfeatures the model correctly predicts.

We can also score the perceptron, using Perceptron.score(). This just returns the accuracy, without updating the weights.

So after one epoch, the perceptron performs well on the verbs it has seen already.

So far we’ve trained the model in batch, which means it has seen all the data once before we score it.

4.2 Second Stage: Medium Frequency Verbs

4.4 Putting it all together

5 Extension: Error Analysis

The next question is less structured and will be marked harder. To reiterate, you can get an Awithout even attempting this question.

Some of the following questions will require you to write code, in this case use the relevant cells labelled # YOUR CODE HERE for question xxx. For text answers use the cell labelled YOUR ANSWERHERE for question xxx. For some questions it is helpful to experiment with the provided code.

You can modify it where it’s provided or add extra code cells to experiment. You are permitted toadd additional cells in order to add written explanations or code if needed.

Now we will analyse the effect of the receptive fields we considered in question 6 using an image. To this end, we select an image from the ImageNet dataset and convert it to grey scale.

Examine the code below and describe how the image is manipulated. Explain what effect we can simulate with this manipulation (see lectures).

# YOUR CODE HERE

raise NotImplementedError()

1. Filter the output of the first convolution (preferably use the parameters you obtained in theprevious question) with the receptive field from question 6c. Show each stage of this process, from raw distorted image to the twice filtered image.