Hello, if you have any need, please feel free to consult us, this is my wechat: wx91due

2 Programming [50 Points]

2.1 Setup

You must create a new virtual environment for the assignment. You can then simply install the requirements by running the command. Make sure your python version and requirement version matches when you install the libraries. 

python -m pip install -r requirements.txt

2.2 Problem Statement

https://www.cs.toronto.edu/∼kriz/cifar.html

CIFAR-10 contains 60000 32x32 colour images in 10 classes (airplane, automobile, bird, cat, deer, dog, frog, horse, ship, truck), with 6000 images per class. There are 50000 training images and 10000 test images.

You will mainly use PyTorch for this assignment. PyTorch is a machine learning framework. You can " nd many useful documentations on their website. You will program a fully connected neural network and a convolutional neural network as well as training/testing using PyTorch and experience hyperparameter search for deep learning models. Notice that, we have written problems in this programming assignment!

There are 3 python " les under the homework 3 coding folder. The data.py has implemented for you but we still recommend you read the code to understand how to prepare Dataloadersfor your  models to train and test on. Please go through models.py and run.py before youbegin! The " les are annotated with TODO in every location where you need to add code and have multiple comments, tips, and recommendations for how to proceed. In addition, you will never need to modify any existing code. After completion, please submit models.py, run.py, cifar-10-fcn.pt and cifar-10-cnn.pt to Gradescope.

2.3 Fully Connected Neural Network, Training and Testing

Your " rst task is to determine the number of input features for the " rst layer and number of output features for the third layer. Notice that, the third layer is the classi" cation layer. Then you need to initialize the layers. Here, we provided the second layer as an example:

After initializing layers, you should complete the forward pass using self.activation_function.

Apply the activation function to the " rst linear layer and the second linear layer.

You can now call this function under if __name__ == "__main__": train_and_test(model_name="fcnet", dataset=CIFAR_10_dataset, num_epochs=5, learning_rate=1e-3, activation_function_name="relu")

Then run python run.py and you should be able to get a test accuracy > 50%. You could also see the labels as well as predictions for 5 images in the CIFAR-10 test set.

2.4 Convolutional Neural Network, Hyperparameter Search

Firstly, you should implement the convolutional neural network in models.py. This time you will additionally use nn.Conv2d, nn.MaxPool2d and nn.Flatten. The " rst convolutional layer (kernel size: 3x3) has 32 output channels, the second convolutional layer (kernel size: 3x3) has 64 output channels. Your " rst task is to determine the number of input channels for the " rst convolutional layer. You should only apply activation function to these 2 convolutionallayers. After 2 convolutional layer, there’s a max pooling layer (kernel size: 2x2) and a # atten layer. Finally after the # atten layer, there’s a linear classi" cation layer which has 12544 input features. To see how we obtain the number 12544, you can refer to CS231n in Stanford, but 

it’s not a requirement for this course. You can alternatively to use Tensor.size after the # atten layer to know the number of input features will be for the linear layer.

After initializing the layers, you can complete the forward pass of the convolutional neu ral network. It should be pretty similar to how you did it for the fully connected neural network.

Additionally, complete grid search on learning rates and activation functions in run.py. Keep the number of epochs to be 5. Hyperparameter Grid search will take around 30 minutes and do not close the computer while doing hyperparameter search. It might stop searching. Record the test accuracy in percentage (round to 1 decimal place) for each hyperparameter con! guration and then complete the table below (see Table. 4).

Finally, report the best test accuracy and the best con! guration of hyperparameters. [5 points]

Describe what you found for learning rates and explain your ! ndings. [5 points]

Additionally, you can " nd code implemented for you under runs.py for saving trained models.

Don’t forget to submit four ! les - models.py, run.py, cifar-10-fcn.pt and cifar-10-cnn.pt to the Gradescope. The Gradescope tests will check whether your fully-connected neural network and convolutional neural network are correct or not. [35 points]