IMSE7140 Assignment 2

2024-11-06 Admin 写评论


Hello, if you have any need, please feel free to consult us, this is my wechat: wx91due


IMSE7140 Assignment 2
Cracking CAPTCHAs
(20 points)

2.1 Brief Introduction

CAPTCHA or captcha is the acronym for “Completely Automated Public Turing test to tell Computers and Humans Apart.” You must have been already familiar with it

because of its popularity in preventing bot attacks or spam everywhere. This assignment, however, will guide you in implementing a deep learning model that can crack a commercial-level captcha!

You deliverables for this assignment should include

1. A single PDF file answers.pdf with answers to all the questions explicitly marked by “Q” with a serial number in this document, and
2. A train.py file to fulfill the programming task requirements marked by “PT.”

Of course, GPUs can facilitate your experiments—Don’t worry if you don’t have any, the training requirement is deliberately simplified.

2.2 Training your model

The captchas we will crack is the multicolorcaptcha. Please pip install the exact version 1.2.0 (the current latest one) in case there might be any incompatibility for other releases.

We use the following codes to generate captchas.
1 from multicolorcaptcha import CaptchaGenerator
2
3 generator = CaptchaGenerator (0)
4 captcha = generator . gen_captcha_image ( difficult_level =0)
5 image = captcha . image
6 characters = captcha . characters
7 image . save ( f"{ characters }. png", "PNG")

In this snippet, CaptchaGenerator(0) configures the image size to 256 × 144 pixels, and the difficult level is set to 0 so that the captchas only contains four 0–9 digits.

Please run the code snippet on your computer. If the captcha is successfully generated, it should look like Figure 2.1.


Figure 2.1: Sample captcha with digits 0570

The training and the validation datasets are generated and attached in folders capts train and capts val. For any machine learning problem, before you start to devise a solution, it is always a good idea to observe the data and gain some intuition first. You may immediately recognize some difficulties in this task:

• The digits have a set of random fonts and colors;
• Some certain range of random rotations are applied to the digits;
• Some line segments are randomly added to the image.

Such a task is considered impossible for traditional pattern recognition methods, which may tackle the problem in a process like this: image thresholding, segmentation, handcrafted filter design, and pattern matching. We can conjecture that “filter design” may fail in capturing useful features and “pattern matching” may have a poor performance.

Fortunately, in the deep learning era, we can delegate the pattern or feature extraction job to deep neural networks. As introduced in the previous lecture “Deep Learning for Computer Vision,” the slide “Understand feature maps: CAPTCHA recognition” shows that a typical architecture for the task consists of two parts:

1. A backbone model to extract a feature map from the captcha image, and
2. A certain amount of prediction heads to interpret the feature map to readable forms.

We will follow this architecture in this assignment. I encourage you to search opensource solutions and learn from their experience. Here we follow this Kaggle post by Ashadullah Shawon.

PT| Use capts train as the training dataset, capts val as the validation dataset, and Keras as the deep learning framework, referring to Shawon’s solution, provide the training code train.py that fulfills the following requirements. “Copy and paste” the codes from the original post is allowed, as well as other AI-generated codes.

1. The maximal number for epochs should be 10. Considering some students will train the model by CPU, it is fair to limit the number of epochs, so the training time for the model should be less than half an hour.
2. The accuracy for one digit should be no less than 30% after training for 10 epochs. The training outputs contain four accuracies respective to the four digits. Since they are similar, you will only need to examine one of them. Keep in mind that 30% for one digit indicates that the overall accuracy for the recognition is only 0.3 4 = 0.81%. Such a low accuracy is not useful for cracking the captcha. However, on the one hand, you may need a GPU to experiment on a practical solution; on the other hand, a wild guess for a 0–9 digit has an accuracy of 10%, so if your model’s accuracy can reach 30% after 10 epochs, it already indicates the model learns from the training set. Hint: if the accuracy for one digit keeps wandering around 0.1 but not increasing in the first two or three epochs, it is the signal that you should modify somewhere in your code and try again.

3. The trained model should be saved as a file my model.keras after training. Though, this model file my model.keras doesn’t need to be uploaded.

Q1| Can we convert the captcha images to grayscale at the preprocessing stage before training? What is the possible advantage by doing that? If any, can you point out the possible disadvantage?
Q2| After the 10-epoch training, what are your accuracies of one digit, for the training and the validation datasets respectively?
Q3| Is the accuracy for the validation dataset lower than that for the training dataset? What are the possible reasons?
Q4| How can we improve the model’s performance on the validation dataset? List at least three different measures.

2.3 Example: A practical model

To demonstrate that the backbone–heads architecture can actually solve the real-world captcha, I trained a relatively large model by an Nvidia GeForce RTX 3090 GPU. You may find in attached the model file 0991-0.9956.keras and the inference code inference.py. The accuracies versus training epochs are shown in Figure 2.2. The inference code reads a randomly generated captcha, inferences the model, and compares the predicted results with the targets. You can press “n” for the next captcha or “q” to quit the program. You may need to pip install keras cv to run the code.
Q5| What kind of backbone did I use in the model 0991-0.9956.keras?
Q6| The backbone’s pre-trained weights on the ImageNet 2012 dataset were loaded before training. What is the possible advantage by doing that?
Q7| Why didn’t I use any dropout in the model? Guess the reason.
Q8| In Figure 2.2, you may have noticed that the accuracies rise very fast from 0 to 0.9, but significantly slow from 0.95 to 0.99. Explain the phenomenon.
Q9| Using the same hardware (which means you can’t upgrade the GPU, for example), how can we speed up the learning process of the model, i.e. the rate of convergence?

Figure 2.2: Accuracies through 1000 epochs in training

Q10| Since the accuracy for one digit is about 99%, the overall accuracy for a captcha is 0.994 ≈ 96%. This performance would be better than humans. Can you propose some methods that can even further improve the performance?

Please note that, not all the questions above have a definite answer. You may also need to do some research as the course doesn’t cover all the details in class. The source code for training this model and the reference answers will be available on Moodle or sent by email after all the students completing the submission.

发表评论

电子邮件地址不会被公开。 必填项已用*标注