Hello, if you have any need, please feel free to consult us, this is my wechat: wx91due

COMP3230 Principles of Operating Systems

Programming Assignment One
Due date: Oct. 17,2024, at 23:59

Total 13 points - Release Candidate Version 2

Programming Exercise- Implement a LLM Chatbot Interface

Objectives

1. An assessment task related tolLO4 [Practicability]-“demonstrate knowledge in applying system software and tools available in the modern operating system for software development".
2. Alearning activity related tolLO 2a.
3. The goals of this programming exercise are:

To have hands-on practice in designing and developing a chatbot program, which involves the creation, management and coordination of processes.

To leam how to use various important Unix system functions:

• to perform process creation and programexecution
• to support interaction between processes by using signals and pipes
• to get the processes's running status by reading the/proc file system
• to configure the scheduling policy of the process via syscall

Tasks

Chatbots like ChatGPTor Poe are the most common user interfaces to large language models (LLMs). Compared with standalone inference programs, it provides a natural way to interact with LLMs.For example, after you enter "What is Fibonacd Number" and press Enter, the chatbot will base on your prompt and use LLM to generate, for example, "Fibonacci Number is a series of numbers whose value is sum of previous two...". But it's not the end, you could further enter prompt like "Write a Python program to generate Fibonacci Numbers." And the model would continue to generate based on the previous messages like "def fibonacci_sequence(n): ...".

Moreover, in practice, we usually separate the inference process handles LLM from main process that handles user input and output, which leads to a separable design that facilitates in-depth control on inference process. For example, we can observe the status of the running process via reading the /proc file system or even control the scheduling policy of the inference process from the main process via relevant syscall.

Though understanding GPT structure is not required, in this assignment, we use Llama3, an open- source variation of GPT and we provide a complete single-thread LLM inference engine as the startpoint of your work. You need to use Unix Process API to create inference process that runsLLM,use pipe and signal to  communicate between two processes, read /proc pseudo file system to monitor running status of the inference process, and use sched syscall to set the scheduler of the inference process and observe the performance changes.

Acknowledgement: The inference framework used in this assignment is based on the open-source project llama2.c by Andrej Karpathy. The LLM used in this assignment is based on SmolLM by HuggingfaceTB.Thanks open-source!