Hello, if you have any need, please feel free to consult us, this is my wechat: wx91due
[CS8803-SMR] Special Topics:
Systems for Machine Learning
Course Description
The new resurgence and efficacy of Machine Learning systems is fueled by progress on Machine Learning algorithms as well as advances in hardware and software systems that support them. Examples of this relationship can be found in enabling training increasingly complex models on growing datasets with ever-improving time-to-convergence and time-to-accuracy. New software systems have also contributed to the simplification of model development, neural architecture discovery, and model fine-tuning, by providing practical abstractions and fueling rapid growth in the machine learning community. These new software systems include open source frameworks such as PyTorch, Tensorflow, MXNet, Clipper, Ray. New hardware platforms specialized for Machine Learning include new generations of GPUs as well as hardware accelerators, such as Google’s TPU and Intel’s Nervana Neural Network Processor (NNP). In this course we will look at the latest trends on the intersection of these three disciplines of Computer Science: Machine Learning, Software Systems, and Hardware Systems. We will introduce the newly emergent research area of Systems for Machine Learning and learn what constitutes good (and bad) research in this area.
The list of topics covered in this class will include:
● ML lifecycle management,
● ML model serving/inference,
● frameworks for ML training and inference,
● latency aware neural architecture search,
● federated learning,
● weight shared Deep Neural Network (DNN) training
● ML model hyperparameter optimization
● Resource management & scheduling for ML workloads
The course format is a mixture of lecture material presented by the instructor and assigned paper reviews presented by the students. The course is heavily project-based by design. To learn to be a SysML researcher --- there’s no substitute for practice. Students will be responsible for paper readings and a hands-on course project. Students will be strongly encouraged to team up in a way that diversifies their expertise, producing full coverage of both Systems and ML background needed for the execution of your project proposals ( e.g., by including both ML and Systems students in each group).
Note: this course counts as an elective towards the MSCS Specialization in ML [source] as well as a PhD qualifier course for the following research areas: SCS Systems.
Prerequisites/Requirements
Fundamentally, you need to have a strong background in at least one of {Systems, ML}:
● Basic system building skills are expected (at the level of CS2200 OR ECE3058)
○ Knowledge of python is required
○ Knowledge of C++ is preferred, but not required
● Basic familiarity with Machine Learning training and/or inference is expected
○ A crash course on Deep Neural Networks (DNN) is strongly recommended, but not required.
● Ability to work with a medium-sized code base (1000-10000 lines of code) is strongly recommended
Academic Honor Code
All students must follow the academic integrity and Georgia Tech Honor Code. Cheating will not be tolerated. Examples of behaviors that violate Georgia Tech Honor Code (Section 3) include but are not limited to:
● Unauthorized collaboration -- this includes copying paper reviews, having a student from a different project group make significant/tangible contributions to the project you are claiming credit for
● Plagiarism: submission of material that is significantly identical to that created or published by another person without adequate credit
● False Claims of Performance: false or exuberant claims of experimental evaluation in a project report that cannot be reproduced with the submitted code.
● Use of ChatGPT or other generative AI technology without attribution to the source!
Late Penalty
A late penalty on assignments will be assessed at 10% point reduction per day. For paper review submissions (if graded), lowest 20% of paper review scores will be dropped for each student.
Research Project
Research project is the major contributor to the student’s grade in this course (see grading policy). Students are expected to work in teams, develop a research idea in the scope of the SysML research area covered by this class, to implement the system prototype, develop experimental methodology, carry out experiments, and communicate the results of their research to the class. The research project will have multiple graded components, including:
● Research project proposal -- initial project proposal for the research project, team mates, falsifiable hypothesis, and experimental methodology
● Mid-term progress report -- project progress report
● Final project report
● Research project poster/demo/video
Labs
Labs are provided as an alternative to the research project. There will be four labs in this class:
Lab 1: 10%
Lab 2: 15%
Lab 3: 15%
Lab 4: 15%
The number of students who can take the lab option will be limited to 15-20 students total.
Final Exam
This course has no midterm or final exam. In lieu of the allocated final exam time slot (see schedule for the date and time) students may be expected to present their final project poster (TBA). The final exam time slot for this course will be accessible on the course’s tentative schedule.
Grading Scale
Your final grade will be assigned as a letter grade according to the following scale:
A 90-100%
B 80-89%
C 70-79%
D 60-69%
F 0-59%
Grading Policy
The following graded assignment will contribute to the student’s final grade.
Class participation: 10%
● 10% -- Attendance / Class participation/Info card
Analytical Paper Presentations: 10%
● 10% -- Paper presentations (submitted to canvas)
In-class quizzes on lectures + papers
● 25% -- administered in class via canvas quiz assignments
Research Project xor Labs: 55%
● Research Project: 55%
○ 5% project proposal
○ 10% mid-point presentation
○ 10% final project presentation
○ 5% final project poster/video/demo
○ 5% team project check-in
○ 20% final project report
● Labs:
○ Lab1: 10%
○ Lab2: 15%
○ Lab3: 15%
○ Lab4: 15%
Communication
This term we will be using Piazza for class discussion. The system is highly catered to getting you help fast and efficiently from classmates, the TA, and myself. Rather than emailing questions to the teaching staff, I encourage you to post your questions on Piazza. If you have any problems or feedback for the developers, email [email protected].
Find our class signup link at: https://piazza.com/gatech/fall2023/cs8803smr
Subject to Change Statement
Due to the highly dynamic situation (e.g., global phenomena outside of the instructor’s control, conference travel, etc), the syllabus and course schedule may be subject to change. It is the responsibility of students to check Canvas, Piazza, email messages, and course announcements (through course canvas and piazza) to stay up-to-date with any course logistics changes. We will make every effort to communicate changes via these mechanisms. The course is held IN RESIDENCE until and unless announced otherwise on course Piazza or Canvas. Virtual option will not be available this semester.
Audit Policy
A student interested in registering in audit mode must always approach the course instructor and determine the minimum course audit requirements. For this class, minimum course audit requirements include full participation in the course project ( including proposal, mid-point presentation, and final project deliverables) OR labs. That’s 55% of the course grade. Therefore, audit and pass/fail are highly discouraged, because the amount of work involved will be similar to taking the course for a letter grade.