Hello, if you have any need, please feel free to consult us, this is my wechat: wx91due

DDA3020 Homework 1

Instructions

1 Written Problems (50 points)

1.1. (Learning of Linear Regression, 25 points)

iii) (8 pts) We further suppose that x1, x2, . . . , xN are drawn from N (µ, σ² ). Show that the maximum likelihood estimation  

1.2. (Support Vector Machine, 25 points) Given two positive samples x1 = (3, 3)T , x2 = (4, 3)T , and one negative sample x3 = (1, 1)T , find the maximum-margin separating hyperplane and support vectors.

Solution steps:

2 Programming (50 points)

2.1. (Linear regression, 25 points)

We have a labeled dataset D = {(x1, y1),(x2, y2), · · · ,(xn, yn)}, with xi ∈ R d being the d-dimensional feature vector of the i-th sample, and yi ∈ R being real valued target (label).

A linear regression model is give by

(1)

where w0 is often called bias and w1, w2, . . . , wd are often called coefficients.

Now, we want to utilize the dataset D to build a linear model based on linear regression. We provide a training set Dtrain that includes 2024 labeled samples with 11 features (See linear regression train.txt) to fit model, and a test set Dtest that includes 10 unlabeled samples with 11 features (see linear regression test.txt) to estimate model.

Datasets You are provided with the iris dataset. The data set contains 3 classes of 50 instances each, where each class refers to a type of iris plant. There are four features: 1. sepal length in cm; 2. sepal width in cm; 3. petal length in cm; 4. petal width in cm. You need to use these features to classify each iris plant as one of the three possible types.

What you should do You should use the SVM function from python sklearn package, which provides various forms of SVM functions. For multiclass SVM you should use the one vs rest strategy. You are recommended to use sklearn.svm.svc() function. You can use numpy for vector manipulation. For technical report, you should report the results required as mentioned below (e.g. training error, testing error, and so on).

If we view the one vs all strategy as combining the multiple different SVM, each one being a separating hyperplane for one class and the rest of the points, then the w, b and support vector indices for that class is the corresponding parameters for the SVM separating this class and the rest of the points. If a variable is of vector form, say   then you should writeeach entry in the same line with comma separated e.g. [1,2,3].

3. (6 points) Calculation using SVM with Slack Variables (Linear Kernel). For eachC = 0.25 × t, where t = 1, 2, . . . , 4, train your SVM on the training dataset, and subsequentlyvalidate it on the testing dataset. Calculate the classification error for both the training andtesting datasets, the weight vector w, the bias b, and the indices of support vectors, and theslack variable ζ of support vectors (you may compute it as max(0, 1 − y · f(X)). The output format is as follows:

Submit your executable code in a “HW1 yourID Q2.ipynb” Jupyter notebook(”.py” file is also acceptable). Indicate the corresponding question number in the comment for each cell, and ensure that your code can logically produce the required results for each question in the required format. Please note that you need to write clear comments and use appropriate function/variable names. Excessively unreadable code may result in point deductions.