Gemini API’s Python SDK can be used for various forms of evaluation, including:
Below is an example of using the LLM to enhance text quality through feedback and grading.
Install the Google GenAI SDK from npm.
You can create your API key using Google AI Studio with a single click.
Remember to treat your API key like a password. Don’t accidentally save it in a notebook or source file you later commit to GitHub. In this notebook we will be storing the API key in a .env file. You can also set it as an environment variable or use a secret manager.
Here’s how to set it up in a .env file:
To load the API key from the .env file, we will use the dotenv package. This package loads environment variables from a .env file into process.env.
Then, we can load the API key in our code:
const dotenv = require("dotenv") as typeof import("dotenv");
dotenv.config({
path: "../../.env",
});
const GEMINI_API_KEY = process.env.GEMINI_API_KEY ?? "";
if (!GEMINI_API_KEY) {
throw new Error("GEMINI_API_KEY is not set in the environment variables");
}
console.log("GEMINI_API_KEY is set in the environment variables");GEMINI_API_KEY is set in the environment variablesIn our particular case the .env is is two directories up from the notebook, hence we need to use ../../ to go up two directories. If the .env file is in the same directory as the notebook, you can omit it altogether.
│
├── .env
└── examples
└── prompting
└── Basic_Evaluation.ipynbWith the new SDK, now you only need to initialize a client with you API key (or OAuth if using Vertex AI). The model is now set in each call.
Now select the model you want to use in this guide, either by selecting one in the list or writing it down. Keep in mind that some models, like the 2.5 ones are thinking models and thus take slightly more time to respond (cf. thinking notebook for more details and in particular learn how to switch the thiking off).
Start by defining some system instructions for this problem. For demonstration purposes, the use case involves prompting the model to write an essay with some mistakes. Remember that for generation tasks like writing an essay, you can change the temperature of the model to get more creative answers.
const STUDENT_SYSTEM_PROMPT = `
You're a college student. Your job is to write an essay riddled with common mistakes and a few major ones.
The essay should have mistakes regarding clarity, grammar, argumentation, and vocabulary.
Ensure your essay includes a clear thesis statement. You should write only an essay, so do not include any notes.
`;
const student_response = await ai.models.generateContent({
model: MODEL_ID,
contents: "Write an essay about the benefits of reading.",
config: {
systemInstruction: STUDENT_SYSTEM_PROMPT,
},
});
tslab.display.markdown(student_response.text ?? "");In today’s fast-paced, high-tech world, many people wonder if reading is still relevant. But let me tell you, reading, like books and articles and stuff, is super beneficial for people, helping them in many ways, like getting smarter and feeling better. It’s really important for your brain and mind, and just generally makes you a more capable person. This essay will explain why reading is so good for us, covering its intellectual, emotional, and social upsides.
First of all, reading makes your brain better and gives you lots of information. When you read, your brain works hard, which builds up your cognitive prowess, making you more intelligent over time. It helps with memory and also focus, because you have to concentrate on what you are reading, this is a very good thing for anyone. For instance, reading complex novels helps you think critically and solve problems, it expands your vocabulary too. People who read a lot often know more things, they are smarter, and can understand difficult concepts easier. It’s like a workout for your brain, making it stronger and more capable of learning new stuff, which is a major advantage in life.
Furthermore, reading has a great effect on our emotional and mental well-being. It can be a truly therapeutic activity, helping people relax and reduce stress after a hard day. Losing yourself in a good story is proven to significantly reduce stress levels, sometimes even more effectively then listening to music. Moreover, reading helps you develop empathy because you get to see the world from different characters’ perspectives. You learn about different cultures, different lives, and this makes you a more understanding person. It really affects how you relate to others, making you kinder and more empathetic, which is a big deal in our society where sometimes people are not very nice to each other. This also helps with social skills, like knowing what to say.
Beyond the personal benefits, reading also has social and practical advantages. People who read lots often communicate better, both in speaking and writing, because they’ve seen how good writers do it. This is a very important skill for getting a good job or just doing well in school. Plus, reading exposes you to new ideas and concepts, broadening your horizons and making you more knowledgeable about the world. You learn about history, science, current events, and all sorts of things you might not learn otherwise. Therefore, reading makes you more informed, which is indispensable for participating in discussions and making good decisions in life. It gives you a deeper comprehension of the diverse world we inhabit, which everyone needs.
In conclusion, it is clear that reading offers a vast array of benefits that positively impact individuals in numerous ways. From enhancing cognitive functions and making you smarter, to promoting emotional well-being and reducing stress, reading is profoundly impactful. It also improves communication abilities and broadens one’s understanding of the world around them, making them more informed and capable citizens. Because of all these important reasons, everyone should really try to read more, as it truly is a powerful tool for personal growth and societal betterment, and it’s also fun.
const TEACHER_SYSTEM_PROMPT = `
As a teacher, you are tasked with grading students' essays.
Please follow these instructions for evaluation:
1. Evaluate the essay on a scale of 1-5 based on the following criteria:
- Thesis statement,
- Clarity and precision of language,
- Grammar and punctuation,
- Argumentation
2. Write a corrected version of the essay, addressing any identified issues
in the original submission. Point out what changes were made.
`;
const teacher_response = await ai.models.generateContent({
model: MODEL_ID,
contents: student_response.text ?? "",
config: {
systemInstruction: TEACHER_SYSTEM_PROMPT,
},
});
tslab.display.markdown(teacher_response.text ?? "");Here’s an evaluation of your essay, followed by a corrected version.
1. Evaluation Scale (1-5):
Overall Score: 3/5
The essay presents a clear stance and is well-organized, but its effectiveness is significantly hampered by informal language, grammatical errors, and a lack of depth in its argumentation.
Here is a corrected version of your essay, with changes highlighted and explained.
Original Title: (Implicitly: The Relevance of Reading)
Corrected Title: The Enduring Relevance and Benefits of Reading
Original Paragraph 1: In today’s fast-paced, high-tech world, many people wonder if reading is still relevant. But let me tell you, reading, like books and articles and stuff, is super beneficial for people, helping them in many ways, like getting smarter and feeling better. It’s really important for your brain and mind, and just generally makes you a more capable person. This essay will explain why reading is so good for us, covering its intellectual, emotional, and social upsides.
Corrected Paragraph 1: In today’s fast-paced, high-tech world, many people wonder if reading retains its relevance. [Change 1: “retains its relevance” is more formal than “is still relevant.”] However, reading offers profound benefits for individuals, fostering intellectual growth, enhancing emotional well-being, and strengthening social capabilities. [Change 2: Replaced informal phrases like “But let me tell you,” “like books and articles and stuff,” “super beneficial,” “getting smarter and feeling better,” “really important,” “so good,” and “upsides” with more precise and academic language. The sentence is also more concise and impactful.] This essay will explore these multifaceted advantages, demonstrating why reading remains an indispensable activity. [Change 3: Refined the thesis statement for a more formal and direct tone, using “multifaceted advantages” and “indispensable activity.”]
Original Paragraph 2: First of all, reading makes your brain better and gives you lots of information. When you read, your brain works hard, which builds up your cognitive prowess, making you more intelligent over time. It helps with memory and also focus, because you have to concentrate on what you are reading, this is a very good thing for anyone. For instance, reading complex novels helps you think critically and solve problems, it expands your vocabulary too. People who read a lot often know more things, they are smarter, and can understand difficult concepts easier. It’s like a workout for your brain, making it stronger and more capable of learning new stuff, which is a major advantage in life.
Corrected Paragraph 2: First, reading significantly enhances cognitive functions and broadens one’s knowledge base. [Change 4: Replaced informal “makes your brain better and gives you lots of information” with more academic phrasing.] Engaging with text actively stimulates the brain, thereby improving cognitive prowess and fostering long-term intelligence. [Change 5: Improved sentence structure and word choice. “Actively stimulates” and “fostering long-term intelligence” are more precise.] It specifically aids in strengthening memory and improving focus, as sustained concentration is required. [Change 6: Corrected the run-on sentence by separating the ideas and making the connection clearer.] For instance, grappling with complex novels cultivates critical thinking and problem-solving skills, in addition to expanding one’s vocabulary. [Change 7: Corrected the comma splice/run-on sentence. “Grappling with” is more active than “reading.” “In addition to” is more formal than “too.”] Individuals who read extensively often possess a wider general knowledge and can more readily grasp intricate concepts. [Change 8: Replaced “read a lot” with “read extensively” and rephrased “know more things, they are smarter” for better clarity and formality.] This mental exercise strengthens the brain, making it more adaptable and efficient at absorbing new information, a crucial asset in any domain. [Change 9: Replaced “new stuff” with “new information” and “major advantage in life” with “crucial asset in any domain” for a more formal tone.]
Original Paragraph 3: Furthermore, reading has a great effect on our emotional and mental well-being. It can be a truly therapeutic activity, helping people relax and reduce stress after a hard day. Losing yourself in a good story is proven to significantly reduce stress levels, sometimes even more effectively then listening to music. Moreover, reading helps you develop empathy because you get to see the world from different characters’ perspectives. You learn about different cultures, different lives, and this makes you a more understanding person. It really affects how you relate to others, making you kinder and more empathetic, which is a big deal in our society where sometimes people are not very nice to each other. This also helps with social skills, like knowing what to say.
Corrected Paragraph 3: Furthermore, reading profoundly impacts our emotional and mental well-being. [Change 10: Replaced “great effect” with “profoundly impacts” for a stronger, more academic tone.] It serves as a truly therapeutic activity, aiding relaxation and stress reduction after a demanding day. [Change 11: Minor rephrasing for conciseness.] Immersing oneself in a captivating story is proven to significantly alleviate stress levels, often more effectively than listening to music. [Change 12: Corrected “then” to “than” and used “immersing oneself in a captivating story” for “losing yourself in a good story.”] Moreover, reading cultivates empathy by allowing individuals to experience the world from diverse characters’ perspectives. [Change 13: Used “cultivates empathy” for “helps you develop empathy” and “diverse characters’ perspectives” for “different characters’ perspectives.”] This exposure to varied cultures and lifestyles fosters a more understanding and compassionate individual. [Change 14: Combined and rephrased “You learn about different cultures, different lives, and this makes you a more understanding person” for better flow and formality.] Such experiences enhance one’s ability to relate to others, promoting kindness and empathy—qualities vital for fostering a more harmonious society. [Change 15: Rephrased “a big deal in our society where sometimes people are not very nice to each other” to a more formal and impactful statement about societal harmony. Removed the vague “This also helps with social skills, like knowing what to say” for better focus.]
Original Paragraph 4: Beyond the personal benefits, reading also has social and practical advantages. People who read lots often communicate better, both in speaking and writing, because they’ve seen how good writers do it. This is a very important skill for getting a good job or just doing well in school. Plus, reading exposes you to new ideas and concepts, broadening your horizons and making you more knowledgeable about the world. You learn about history, science, current events, and all sorts of things you might not learn otherwise. Therefore, reading makes you more informed, which is indispensable for participating in discussions and making good decisions in life. It gives you a deeper comprehension of the diverse world we inhabit, which everyone needs.
Corrected Paragraph 4: Beyond the personal realm, reading also confers significant social and practical advantages. [Change 16: “Confers significant” is more academic than “has.”] Individuals who read extensively frequently exhibit superior communication skills, both written and verbal, by internalizing the structures and techniques of accomplished authors. [Change 17: Replaced “read lots” with “read extensively” and “they’ve seen how good writers do it” with “by internalizing the structures and techniques of accomplished authors” for more precision.] This proficiency is invaluable for academic success and professional advancement. [Change 18: Rephrased “getting a good job or just doing well in school” for a more formal tone.] Furthermore, reading exposes individuals to novel ideas and concepts, thereby broadening their horizons and deepening their understanding of the world. [Change 19: Used “novel ideas” for “new ideas” and “deepening their understanding” for “making you more knowledgeable.”] It provides access to knowledge spanning history, science, current events, and diverse subjects that might otherwise remain unexplored. [Change 20: Replaced “all sorts of things” with “diverse subjects.”] Consequently, reading fosters an informed citizenry, essential for engaging in meaningful discussions and making sound life decisions. [Change 21: Improved flow and precision in the statement about being informed and making decisions.] It cultivates a profound comprehension of the diverse world we inhabit, an imperative for informed global participation. [Change 22: Rephrased “which everyone needs” to emphasize global participation.]
Original Paragraph 5: In conclusion, it is clear that reading offers a vast array of benefits that positively impact individuals in numerous ways. From enhancing cognitive functions and making you smarter, to promoting emotional well-being and reducing stress, reading is profoundly impactful. It also improves communication abilities and broadens one’s understanding of the world around them, making them more informed and capable citizens. Because of all these important reasons, everyone should really try to read more, as it truly is a powerful tool for personal growth and societal betterment, and it’s also fun.
Corrected Paragraph 5: In conclusion, it is evident that reading offers a vast array of benefits that profoundly impact individuals. [Change 23: Replaced “clear” with “evident” and removed redundant “positively impact” and “in numerous ways.”] From enhancing cognitive functions and intellectual acuity to promoting emotional well-being and reducing stress, its influence is extensive and deeply impactful. [Change 24: Replaced “making you smarter” with “intellectual acuity” for a more sophisticated term. Used “extensive and deeply impactful” for “profoundly impactful” to vary phrasing.] Moreover, it refines communication abilities and broadens one’s understanding of the world, fostering informed and capable citizens. [Change 25: Rephrased for conciseness and impact. “Refines” for “improves.”] Therefore, embracing reading is not merely a recommendation but a vital practice for personal growth and the betterment of society. [Change 26: Rephrased “Because of all these important reasons, everyone should really try to read more” to a more assertive and formal call to action. Removed the informal “and it’s also fun” as it detracts from the academic conclusion.]
Be sure to explore other examples of prompting in the repository. Try writing your own prompts for evaluating texts.