2. Few-shot prompting presents a set of high-quality demonstrations, each consisting of both in- put and desired output, on the target task. As the model first sees good examples, it can better understand human intention and criteria for what kinds of answers are wanted.  For example, we can send the following 1-shot prompting message to Llama-3.1-8B-Instruct:

messages  =   [

{"role":    "system",  "content":    [System  Message]},

{"role":    "user",  "content":    "Elsa  has  5  apples.   Anna  has  2 more  apples  than  Elsa . How many  apples  do  they  have  together?"},

{  "role":    "assistant",  "content":    "Anna  has  2 more  apples  than  Elsa .   So  Anna  has 2  +  5  =  7  apples.    Elsa  and  Anna  have  5  +  7  =  12  apples  together.    ####  12"  }

{"role":   "user",  "content":    "If  there  are  3  cars  in  the  parking  lot  and  2 more  cars arrive,  how many  cars  are  in  the  parking  lot?"},

]

The choice of prompting format, interaction method, and decoding parameters will all significantly influence task performance. Now enjoy exploring the magic of prompting with Llama-3.1-8B-Instruct!

2 Task

For your selected task (math or code), begin by implementing the two simplest baselines: zero-shot and few-shot prompting, using only the problem statement with demonstrations (optional). Please re- fer to the example format in Section 1. Then, choose two advanced methods to enhance performance beyond baselines. We present advanced methods that you may choose to implement in Section 2.1and Section 2.2, respectively. You are also encouraged to propose your own designs!

Here, we provide two surveys that offer a comprehensive overview of the development of prompt- ing engineering and the self-correction of LLMs:

• A survey for prompt engineering for autoregressive language models [Survey Link].

• A collection of research papers for LLMs self-correcting [Survey Link]. Note that we only focus on the “Generation-Time Correction” and “Post-hoc Correction” .

2.1 Mathematical Reasoning

Data We use the grade school math dataset, GSM8K, which is one of the most popular datasets for evaluating the mathematical reasoning performance of LLMs. Note that the performance of the LLM is measured using “GSM8K/test.jsonl” .  You can utilize “GSM8K/train.jsonl” for other objectives, such as retrieving the most similar examples for the questions in the test file.

Metric For each method, report the overall accuracy on the 1,319 questions in the test file.  Use the answer extraction function provided in “GSM8K/evaluation.py” for calculating accuracy.

Task 1: Implement Baseline We consider two baselines:

• Zero-shot prompt the model using only the problem statement.

• Few-shot prompts the model with a few demonstrations, each containing manually written (or model-generated) high-quality reasoning chains.

We provide the prompting template in “GSM8K/baseline .py” .

Task 2: Implement two Advanced Prompting Methods You may explore various advanced strate- gies for improvement, such as retrieving similar demonstrations, decomposing responses, employing response ensemble, engaging in self-improvement, and more.  Below, we present several advanced methods, although not exhaustive, that you may choose to implement, considering the characteristics of mathematical reasoning tasks. You are also encouraged to propose your own designs!

• Self-Polish: Enhance Reasoning in Large Language Models via Problem Refinement [paper]

• Tree Prompting: Efficient Task Adaptation without Fine-Tuning [paper]

• SELF-REFINE: Iterative Refinement with Self-Feedback [paper]

• PHP: Progressive-Hint Prompting Improves Reasoning in Large Language Models [paper]

• CRITIC: Large Language Models Can Self-Correct with Tool-Interactive Critiquing [paper]

• Teaching Algorithmic Reasoning via In-context Learning [paper]

• Contrastive Decoding Improves Reasoning in Large Language Models paper]

• Self-Evaluation Guided Beam Search for Reasoning [paper]

• Skills-in-Context Prompting: Unlocking Compositionalityin Large Language Models [paper]

• Program of Thoughts Prompting:  Disentangling Computation from Reasoning for Numerical Reasoning Tasks [paper]

• Large Language Models are Better Reasoners with Self-Verification [paper]

Task 3: Combine the Two Methods or Enhance One Method Can you combine the two advanced methods or enhance one method based on your analysis to achieve greater gains? Are complementary or enhanced effects achievable? If not, please explain why.

2.2 Code Generation

A particularly intriguing application of LLMs is code generation,  a task that involves producing source code from natural language descriptions.   This area has garnered substantial interest from both academia and industry, as evidenced by the development of tools like GitHub Copilot.

Data We use HumanEval leaderboard, which has been established as a de facto standard for eval- uating the coding proficiency of LLMs.  Note that the performance of the LLM is measured using “HumanEval/HumanEval.jsonl” .

Metric For each method, report the overall accuracy on the 164 questions in the “HumanEval.jsonl” file. We provided “example_problem.jsonl” and “example_solutions.jsonl” under HumanEval to illustrate the format and assist with debugging. You can evaluate the accuracy of the generated code by running:

python  evaluate_functional_correctness .py  –sample_file  example_samples.jsonl  –problem_file example_problem.jsonl

Task 1: Implement Baseline We use zero-shot prompting as the baseline.  The prompting template is provided in “HumanEval/baseline .py” .

Task 2: Implement two Advanced Prompting Methods You may explore various advanced strate- gies for improvement.  Below, we present several advanced methods that you may choose to imple- ment. You are also encouraged to propose your own designs!

• Teaching Large Language Models to Self-Debug [paper]

• SelfEvolve: A Code Evolution Framework via Large Language Models [paper]

• Self-Edit: Fault-Aware Code Editor for Code Generation [paper]

• Lever: Learning to verify language-to-code generation with execution [paper]

• Is Self-Repair a Silver Bullet for Code Generation? [paper]

• Codet: Code generation with generated tests [paper]

• SELF-REFINE: Iterative Refinement with Self-Feedback [paper]

• Language agent tree search unifies reasoning acting and planning in language models [paper]

Task 3: Combine the Two Methods or Enhance One Method Can you combine the two advanced methods or enhance one method based on your analysis to achieve greater gains? Are complementary or enhanced effects achievable? If not, please explain why.

2.3 Model

In this assignment, we will use Llama-3.1-8B-Instruct, which is a powerful open-source model that natively supports multilingual capabilities, coding, reasoning, and tool usage. For more details about this model, you can refer to the Meta Blog: https://ai.meta.com/blog/meta-llama-3-1/and https: //www.llama.com/docs/model-cards-and-prompt-formats/llama3_1/ .

3 API

In this assignment, you will interact with the Llama-3.1-8B-Instruct sponsored by SambaNova Sys- tem.  To access this resource, please refer to the instructions in the “SambaNova  Cloud  QuickStart  Guide.pdf” to register and generate your API key. To verify that your API key is functioning correctly, you can either use the provided curl command in the document or run the “test_full_response .py” script.

4 Report

You will write a report including the following parts:

• The description of your implemented methods, including the architecture, the hyperparameters, etc.

• The discussion and table of your results, including the hyperparameters you have tested, the performance of the methods on the test set, the analysis of the results, the advantages compared to the baselines, etc.