Important note: the due date is 23/02/2025. You should submit your scripts to the folder in CANVAS. Late submission is not allowed unless it is well justified. Please include the MATLAB code or Python Code as attachment if computer experiment is involved.

Consider the signal-flow graph of the perceptron shown in the above figure. The activation function, ϕ(v)  , where v is the induced local field, can be designed by the user. If the activation function is chosen as hard limiter (i.e. step function), then it becomes the classical perceptron, and the decision boundary is shown to be a hyperplane. In this problem, let’s explore other choices of the activation function, and its effect on the decision boundary. Let’s assume that the classification decision made by the perceptron is simply a threshold rule defined as follows:

whereξis a user-defined threshold; otherwise, x belongs to class C2.

Consider the following three choices of activation function:

1) The activation function is the logistic function:;

2) The activation function is the Bell-shaped Gaussian function:;

Consider the logic function, EXCLUSIVE OR (XOR).

Truth Table of XOR

x1	0	1	0	1
x2	0	0	1	1
y	0	1	1	0

It is well known that the XOR problem is not linearly separable. It seems obvious by visually checking, which however cannot be accepted as mathematical proof.

Therefore, please supply a rigorous mathematical proof for this statement.

Q3. (10 Marks)

The perceptron could be used to perform numerous logic functions, such as AND, OR, COMPLEMENT and NAND function, whose truth tables are tabulated as follows respectively.

a). Demonstrate the implementation of the logic functions AND, OR, COMPLEMENT and NAND with selection of weights by off-line calculations.

(3 Marks)

b). Demonstrate the implementation of the logic functions AND, OR, COMPLEMENT and NAND with selection of weights by learning procedure. Suppose initial weights are chosen randomly and learning rate is 1.0. Plot out the trajectories of the weights for each case. Compare the results with those obtained in (a). Try other learning rates, and report your observations with different learning rates.

(4 Marks)

c). What would happen if the perceptron is applied to implement the EXCLUSIVE OR function with selection of weights by learning procedure? Suppose initial weight is chosen randomly and learning rate is 1.0. Do the computer experiment and explain your finding. (3 Marks)

ANDQ4. (10 Marks)

Single layer perceptron with pure linear activation function can be used to fit a linear model to a set of input-output pairs. Suppose that we are given the following pairs:

{(0,0.5), (0.8, 1), (1.6, 4), (3, 5), (4.0, 6), (5.0, 8)}

and a single linear neuron as shown in the following figure.

a). Find the solution of w and b using the standard linear least-squares (LLS) method.

Plot out the fitting result. (3 Marks)

b). Suppose that initial weight is chosen randomly and learning rate is 0.01. Find the solution of w and b using the least-mean-square (LMS) algorithm for 100 epochs. Plot out the fitting result and the trajectories of the weights versus learning steps. Will the weights converge?

(3 Marks)

c). Compare the results obtained by LLS and the LMS methods.

(2 Marks)

d) Repeat the simulation study in b) with different learning rates , and explain your findings.

(2 Marks)

Q5. (10 Marks)

Consider that we are trying to fit a linear model to a set of input-output pairs (x(1), d(1)), (x(2), d(2)) …, (x(n), d(n)) observed in an interval of duration n, where input x is m-dimensional vector, xxx x = [ 1 2  m ] . T The linear model takes the following form: